检索结果-内蒙古大学图书馆

Analysis of Hadoop log file in an environment for dynamic detection of threats using machine learning

Measurement: Sensors 2022年 24卷

作者： Naidu, K Bapayya Ravi Prasad, B. Hassen, Samar Mansour Kaur, Chamandeep Al Ansari, Mohammed Saleh Vinod, R. Nivetha, M. Kiran Bala, B. Surampalem Jawaharlal Nehru Technological University Kakinada India CSE Dept Marri Laxman Reddy Institute of Technology and Management Hyderabad500043 India Computer Science JazanUnivertsity Jazan Saudi Arabia Jazan University Saudi Arabia Jazan Saudi Arabia College of Engineering Department of Chemical Engineering University of Bahrain Bahrain School of Contemporary Knowledge Systems Chinmaya Vishwavidyapeeth Cochin Kerala India Department of Artificial Intelligence and Data Science K.Ramakrishnan College of Engineering Trichy India Head of the Department Department of Artificial Intelligence and Data Science K.Ramakrishnan College of Engineering Tamil Nadu Trichy India

Log files are a significant piece of records that once properly analyzed, can yield helpful information. The usage of cloud services for log analysis is almost inevitable due to the greater rate of production and quantity of hardware and sensors that produce logs. The idea is to adopt machine learning mechanisms to "understand" what such a test suite's "anticipated" behavior is. This implies the system needs to be able to learn the difference among a Hadoop log file that contains a threat that does not really, based on past patterns. Many programs that really can effectively monitor the Hadoop cluster have been developed. The bulk of these tools collect relevant information from every cluster node and send it to be processed. The majority of these diagnostic tools were post-execution techniques commonly. This paper gives an experimental evaluation of the various log analysts used in Hadoop for error monitoring and detection. MapReduce is configured using these files. The algorithms were tested on a parser set of labelled log files, and their efficiency was assessed by looking for abnormal events in the Hadoop log files and executing an experimentation using the methods. The algorithms used were Term Frequency Inverse Document Frequency, Random Forest, and Local Outlier Factor. Later, the clustering with K-Means and Principal component analysis is utilized to extract certain relevant details from the data, monitor groups of bits of data to find aberrant events. The Term Frequency Inverse Document Frequency strategy surpassed the other two techniques in predicting threat occurrences in data, according to the findings. The findings would help developers in locating abnormal occurrences without having to go through the Hadoop log file row by row individually. The framework generates events that behave in a different way than the rest of such events in the log and leads the mechanism to fail to work. © 2022 The Authors

关键词： Principal component analysis

来源：评论

学校读者我要写书评

暂无评论

Personalized Recommendation Based On Entity Attributes and Graph Features

Personalized Recommendation Based On Entity Attributes and G...

引用

IEEE International Conference on Big knowledge (ICBK)

作者： Yi Zhu Bingbing Dong Zhiqing Sha School of Information Engineering Yangzhou University Yangzhou China Ministry of Education Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology) China School of Computer Science and Information Engineering Hefei University of Technology Hefei China Instituse of Big Knowledge Science Hefei University of Technology Hefei China

ISBN: (纸本)9781665438599

With the rapid increase in the amount of website data, it has been a more difficult task for users to get the infor-mation they are interested in. Personalized recommendation is an important bridge to find the information which users really need on the website. Many recent studies have introduced additional attribute information about users and/or items to the rating matrix for alleviating the problem of data sparsity. In order to make full use of the attribute information and scoring matrix, deep learning based recommendation methods are proposed, especially the autoencoder model has attracted much attention because of its strong ability to learn hidden features. However, most of the existing autoencoder- based models require that the dimension of the input layer is equal to the dimension of the output layer, which may increase model complexity and certain information loss when using attribute information. In addition, as users' awareness of privacy protection increases, user attribute information is difficult to obtain. To address the above problems, in this paper, we propose a hybrid personalized recommendation model, which uses a semi-autoencoder to jointly embed the item's score vector and internal graph features (short for Co-Agpre). Specifically, we regard the user-item historical interaction matrix as a bipartite graph, and the Laplacian of the user-item co-occurrence graph is utilized to obtain the graph features of the item for solving the problem of sparse attributes. Then a semi-autoencoder is introduced to learn the hidden features of the item and perform rating prediction. The proposed model can flexibly use information from different sources to reduce the complexity of the model. Experiments on two real-world datasets demonstrate the effectiveness of the proposed Co-Agpre compared with state-of-the-art methods.

关键词： Deep learning Privacy Laplace equations Conferences Predictive models Feature extraction Complexity theory

来源：评论

学校读者我要写书评

暂无评论

Learning Representations from Local to Global for Fine-grained Patient Similarity Measuring in Intensive Care Unit

Learning Representations from Local to Global for Fine-grain...

引用

IEEE International Conference on data Mining (ICDM)

作者： Xianli Zhang Buyue Qian Yang Li Zeyu Gao Chong Guan Renzhen Wang Yefeng Zheng Hansen Zheng Chen Li School of Computer Science and Technology Xi’an Jiaotong University Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Beijing Chaoyang Hospital Capital Medical University Tencent Shenzhen Guandong China School of Mathematics and Statistics Xi’an Shaanxi China

Patient similarity measurement is an essential step in discovering clinically meaningful subgroups and building case retrieval systems. Most existing studies implement this procedure using similarity measurement algorithms on the multivariate clinical time-series (input space) or the low-dimensional patient representation (representation space) learned by a representation learning model. However, they either suffer from the adverse effects of irrelevant variables in the data or fail to assess the fine-grained similarity underneath the disease progress. In this paper, we propose a method to measure more fine-grained patient similarity in the state space, where each patient is represented by a series of state representations that reveal the dynamic health status. We discuss three desiderata, including stability, personality, and interpretability, for the state representations, and on this basis, develop a supervised predictive model that learns good state representations for identifying similar patients and predicting patient outcomes. Experimental results on the publicly available dataset MIMIC-III show that our method offers a promising direction for precisely identifying similar patients at the state trajectory level, as well as accurately predicting outcomes.

关键词： Training Representation learning Measurement units Precision medicine MIMICs Predictive models Extraterrestrial measurements

来源：评论

学校读者我要写书评

暂无评论

Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features

arXiv

引用

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at this url. © 2024, CC BY.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques

arXiv

引用

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China School of Cyber Science and Tech. Sun Yat-sen University Shenzhen Campus China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is available at this url. © 2024, CC BY.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Foundation models in bioinformatics

引用

National science Review 2025年第4期 400-418页

作者： Fei Guo Renchu Guan Yaohang Li Qi Liu Xiaowo Wang Can Yang Jianxin Wang Hunan Provincial Key Lab on Bioinformatics School of Computer Science and Engineering Central South University Xiangjiang Laboratory Key Laboratory for Symbol Computation and Knowledge Engineering of the Ministry of Education College of Computer Science and Technology Jilin University Department of Computer Science Old Dominion University School of Life Sciences and Technology Tongji University Department of Automation Tsinghua University Department of Mathematics State Key Laboratory of Molecular Neuroscience and Big Data Bio-Intelligence Lab The Hong Kong University of Science and Technology

With the adoption of foundation models(FMs),artificial intelligence(AI) has become increasingly significant in bioinformatics and has successfully addressed many historical challenges,such as pre-training frameworks,model evaluation and *** demonstrate notable proficiency in managing large-scale,unlabeled datasets,because experimental procedures are costly and labor *** various downstream tasks,FMs have consistently achieved noteworthy results,demonstrating high levels of accuracy in representing biological entities.A new era in computational biology has been ushered in by the application of FMs,focusing on both general and specific biological *** this review,we introduce recent advancements in bioinformatics FMs employed in a variety of downstream tasks,including genomics,transcriptomics,proteomics,drug discovery and single-cell *** aim is to assist scientists in selecting appropriate FMs in bioinformatics,according to four model types:language FMs,vision FMs,graph FMs and multimodal *** addition to understanding molecular landscapes,AI technology can establish the theoretical and practical foundation for continued innovation in molecular biology.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Beware of the simulated DAG! Causal discovery benchmarks may be easy to game

arXiv

引用

arXiv 2021年

作者： Reisach, Alexander G. Seiler, Christof Weichwald, Sebastian Department of Mathematical Sciences University of Copenhagen Denmark Department of Data Science and Knowledge Engineering Maastricht University Netherlands Mathematics Centre Maastricht Maastricht University Netherlands

Simulated DAG models may exhibit properties that, perhaps inadvertently, render their structure identifiable and unexpectedly affect structure learning algorithms. Here, we show that marginal variance tends to increase along the causal order for generically sampled additive noise models. We introduce varsortability as a measure of the agreement between the order of increasing marginal variance and the causal order. For commonly sampled graphs and model parameters, we show that the remarkable performance of some continuous structure learning algorithms can be explained by high varsortability and matched by a simple baseline method. Yet, this performance may not transfer to real-world data where varsortability may be moderate or dependent on the choice of measurement scales. On standardized data, the same algorithms fail to identify the ground-truth DAG or its Markov equivalence class. While standardization removes the pattern in marginal variance, we show that data generating processes that incur high varsortability also leave a distinct covariance pattern that may be exploited even after standardization. Our findings challenge the significance of generic benchmarks with independently drawn parameters. The code is available at https://***/Scriddie/Varsortability. Copyright © 2021, The Authors. All rights reserved.

关键词： Standardization

来源：评论

学校读者我要写书评

暂无评论

Recovering accurate labeling information from partially valid data for effective multi-label learning 29

Recovering accurate labeling information from partially vali...

引用

29th International Joint Conference on Artificial Intelligence, IJCAI 2020

作者： Li, Ximing Wang, Yang College of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation Knowledge Engineering of Ministry of Education Jilin University China Key Laboratory of Knowledge Engineering With Big Data Ministry of Education Hefei University of Technology China School of Computer Sci and Information Engineering Hefei University of Technology China

ISBN: (纸本)9780999241165

Partial Multi-label Learning (PML) aims to induce the multi-label predictor from datasets with noisy supervision, where each training instance is associated with several candidate labels but only partially valid. To address the noisy issue, the existing PML methods basically recover the ground-truth labels by leveraging the ground-truth confidence of the candidate label, i.e., the likelihood of a candidate label being a ground-truth one. However, they neglect the information from non-candidate labels, which potentially contributes to the ground-truth label recovery. In this paper, we propose to recover the ground-truth labels, i.e., estimating the ground-truth confidences, from the label enrichment, composed of the relevance degrees of candidate labels and irrelevance degrees of non-candidate labels. Upon this observation, we further develop a novel two-stage PML method, namely Partial MultiLabel Learning with Label Enrichment-Recovery (PML3ER), where in the first stage, it estimates the label enrichment with unconstrained label propagation, then jointly learns the ground-truth confidence and multi-label predictor given the label enrichment. Experimental results validate that PML3ER outperforms the state-of-the-art PML methods. © 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved.

关键词： Learning systems

来源：评论

学校读者我要写书评

暂无评论

Unboxing the black-box of deep learning based reconstruction of undersampled MRIs 4

Unboxing the black-box of deep learning based reconstruction...

引用

4th Italian Workshop on Explainable Artificial Intelligence, XAI.it 2023

作者： Chatterjee, Soumick Das, Arnab Khatun, Rupali Nürnberger, Andreas Genomics Research Centre Human Technopole Milan Italy Faculty of Computer Science Otto von Guericke University Magdeburg Magdeburg Germany Data and Knowledge Engineering Group Otto von Guericke University Magdeburg Magdeburg Germany German Research Centre for Artificial Intelligence Kaiserslautern Germany Translational Radiobiology Department of Radiation Oncology Universitätsklinikum Erlangen Erlangen Germany Department of Radiation Oncology Universitätsklinikum Erlangen Friedrich-Alexnder-Universität Erlangen-Nürnberg Erlangen Germany Comprehensive Cancer Centre Erlangen-EMN Erlangen Germany Centre for Behavioural Brain Sciences Magdeburg Germany

Deep learning has emerged as a very important area of research and has shown immense potential in solving different kinds of problem, including in the medical field. For tasks like undersampled MRI reconstruction - the process of speeding up MRI acquisition with the help of undersampling, deep learning has shown its dominance over the years. But one of the major problems with deep learning is trust: Complex reasonings done by these models appear black-box to the users. Therefore, to build trust and better acceptability, it is important to open up this black-box nature of these models. For classification models, several approaches have been proposed. Nevertheless, for models dealing with inverse problems, like the reconstruction of the undersampled MRIs, it is more challenging as the output of the model has the same number of output pixels as the input, making the interpretability of such models more complex. This research explores different methods to understand the working mechanism of a deep learning model for the task of undersampled MRI reconstruction. © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

关键词： Inverse problems

来源：评论

学校读者我要写书评

暂无评论

Learning Group-Disentangled Representation for Interpretable Thoracic Pathologic Prediction

Learning Group-Disentangled Representation for Interpretable...

引用

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Hao Li Yirui Wu Hexuan Hu Hu Lu Yong Lai Shaohua Wan Key Laboratory of Water Big Data Technology of Ministry of Water Resources Hohai University College of Computer and Information Hohai University Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University School of Computer Science and Communication Engineering Jiangsu University Shenzhen Institute for Advanced Study University of Electronic Science and Technology of China

ISBN: (数字)9781665468190

ISBN: (纸本)9781665468206

Deep learning methods have shown significant performance in medical image analysis tasks. However, they generally act like ”black box” without explanations in both feature extraction and decision processes, leading to lack of clinical insights and high risk assessments. To aid deep learning in envisioning diseases with visual clues, we propose Representation Group-Disentangling Network (RGD-Net), which can completely disentangle feature space of input X-ray images into several independent feature groups, each corresponding to a specific disease. Taking several semantically related and labeled X-ray images as input, RGD-Net firstly extracts completely group-disentangled representations of diseases through Group-Disentangle Module, which applies group-swap and linking operations to construct latent space by enforcing semantic consistency of attributes. To prevent learning degenerate representations defined as shortcut problem, we further introduce adversarial constricts on mapping from features to diseases, thus avoiding model collapse with former free-form disentanglement. Experiments on chestxray-14 and ChestXpert datasets demonstrate that RGD-Net are effective in predicting diseases with remarkable advantages, which leverage potential factors contributing to different diseases, thus enhancing interpretability in working patterns of deep learning methods.

关键词： Deep learning Training Visualization Image analysis Semantics Feature extraction Risk management

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：