检索结果-内蒙古大学图书馆

Application of Big Data Mining System Integrating spectral clustering algorithm and Apache Spark Framework

INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS 2025年第1期16卷 678-686页

作者： Guo, Yuansheng China Mobile Commun Grp Hunan Co Ltd Changsha 410001 Hunan Peoples R China

spectral clustering algorithm is a highly effective clustering algorithm with broad application prospects in data mining. To improve the efficient data processing capability of big data mining systems, a big data mining system that integrates spectral clustering algorithm and Apache Spark framework is proposed. It is applied in the big data mining system by combining Hdoop, Spark framework, and spectral clustering algorithm. The research results indicated that after 300 iterations of spectral clustering algorithm, the error value tended to stabilize and drops to 0.123. In different datasets, different error values were displayed, indicating that spectral clustering algorithm had better performance in discrete data processing and smaller testing errors. The minimum time consumed by the comparative system was 37.83 seconds, the maximum time was 55.26 seconds, and the average time was 51.65 seconds. The minimum time consumed by the research system was 18.93 seconds, the maximum time consumed was 32.22 seconds, and the average time consumed was 28.14 seconds. Compared with the comparative system, the research system consumed less time, trained faster, and was more conducive to shortening the clustering running time. The algorithm framework and system raised in the research have good operational efficiency and clustering ability in data mining processing, which promotes the reliability and development of big data mining systems.

关键词： spectral clustering algorithm apache spark big data data mining

来源：评论

学校读者我要写书评

暂无评论

spectral clustering algorithm of Uncertain Objects

Spectral Clustering Algorithm of Uncertain Objects

引用

2nd International Conference on Mechatronics and Applied Mechanics (ICMAM2012)2nd International Conference on Mechatronics and Applied Mechanics (ICMAM2012)

作者： He, Tong Cent Univ Finance & Econ Sch Appl Math Beijing 100081 Peoples R China

ISBN: (纸本)9783037856512

By extending classical spectral clustering algorithm, a new clustering algorithm of uncertain objects is proposed in this paper. In the algorithm, each uncertain object is represented as a Gaussian mixture model, and Kullback-Leibler divergence and Bayesian probability are respectively used as similarity measure between Gaussian mixture models. In an extensive experimental evaluation, we not only show the effectiveness and efficiency of the new algorithm and compare it with CLARANS algorithm of uncertain objects.

关键词： Gaussian mixture model Kullback-Leibler divergence Bayesian probability spectral clustering algorithm uncertain objects

来源：评论

学校读者我要写书评

暂无评论

Detecting network communities using regularized spectral clustering algorithm

引用

ARTIFICIAL INTELLIGENCE REVIEW 2014年第4期41卷 579-594页

作者： Huang, Liang Li, Ruixuan Chen, Hong Gu, Xiwu Wen, Kunmei Li, Yuhua Huazhong Univ Sci & Technol Wuhan 430074 Peoples R China Huazhong Agr Univ Wuhan Peoples R China

The progressively scale of online social network leads to the difficulty of traditional algorithms on detecting communities. We introduce an efficient and fast algorithm to detect community structure in social networks. Instead of using the eigenvectors in spectral clustering algorithms, we construct a target function for detecting communities. The whole social network communities will be partitioned by this target function. We also analyze and estimate the generalization error of the algorithm. The performance of the algorithm is compared with the standard spectral clustering algorithm, which is applied to different well-known instances of social networks with a community structure, both computer generated and from the real world. The experimental results demonstrate the effectiveness of the algorithm.

关键词： Community detection Graph laplacian Eigenvector spectral clustering algorithm Regularized spectral clustering algorithm

来源：评论

学校读者我要写书评

暂无评论

spectral clustering algorithm for real-time endpoint detection of silicon nitride plasma etching

引用

PLASMA PROCESSES AND POLYMERS 2023年第6期20卷 e2200238-e2200238页

作者： Lee, Seonghyeon Choi, Hojun Kim, Jaehyeon Chae, Heeyeop Sungkyunkwan Univ SKKU Sch Chem Engn Suwon 16419 South Korea Sungkyunkwan Univ SKKU Sungkyunkwan Adv Inst Nanotechnol SAINT Suwon South Korea

The spectral clustering algorithm (SCA) is developed for endpoint detection (EPD) of Si3N4 plasma etching using optical emission spectroscopy (OES). OES signals are collected in real-time and filtered using discrete wavelet transform (DWT). The SCA using 3648 full-spectrum wavelengths with DWT filtering improves signal-to-noise ratio (SNR) by 2.78 times compared to single-wavelength related to N-2 molecule in 1.0% relative area etching. The wavelengths related to reactants and products are selected to enhance the SNR of the SCA. The SCA using 87 selected wavelengths with DWT filtering improves SNR by 3.57 times compared to SCA using full-spectrum wavelengths. This study demonstrates that the SCA improves the etching EPD sensitivity and can be applied for fault detection of various plasma processes.

关键词： endpoint detection optical emission spectroscopy plasma etching plasma monitoring spectral clustering algorithm

来源：评论

学校读者我要写书评

暂无评论

Efficient parallel spectral clustering algorithm design for large data sets under cloud computing environment

引用

JOURNAL OF CLOUD COMPUTING-ADVANCES SYSTEMS AND APPLICATIONS 2013年第1期2卷 1-10页

作者： Jin, Ran Kou, Chunhai Liu, Ruijuan Li, Yefeng Donghua Univ Coll Informat Sci & Technol Shanghai Peoples R China Zhejiang Wanli Univ Sch Comp Sci & Informat Technol Ningbo Zhejiang Peoples R China

spectral clustering algorithm has proved be more effective than most traditional algorithms in finding clusters. However, its high computational complexity limits its effect in actual application. This paper combines the spectral clustering with MapReduce, through evaluation of sparse matrix eigenvalue and computation of distributed cluster, puts forward the improvement ideas and concrete realization, and thus improves the clustering speed of the distinctive clustering algorithm. According to the experiment, with the processing data scale being enlarged, the clustering rate is in nearly linear growth, and the proposed parallel spectral clustering algorithm is suitable for large data mining. The research results provide research basis to better design a clustering partition algorithm in large data and high efficiency.

关键词： Large data spectral clustering algorithm clustering analysis Parallel Lanczos K-means

来源：评论

学校读者我要写书评

暂无评论

Typical Scene Acquisition Strategy for VPP Based on Multi-scale spectral clustering algorithm 2

Typical Scene Acquisition Strategy for VPP Based on Multi-sc...

引用

2nd IEEE Conference on Energy Internet and Energy System Integration (EI2)

作者： Zhu, Yu Yi, Zhongkai Lu, Qiuyu Liu, Sijie Xu, Yinliang Wang, Kang Guangdong Power Grid Co Ltd Elect Power Dispatching Control Ctr Guangzhou 523000 Guangdong Peoples R China Tsinghua Univ Grad Sch Shenzhen Shenzhen 518055 Peoples R China

ISBN: (纸本)9781538685495

The virtual power plant (VPP) technology can effectively realize the connection of distributed resources and power systems, and is benefit to the safety and economical operation of power systems. This paper considers the value of electric and heat load, fluctuation and changing trend of renewable energy sources such as wind power and photovoltaic. A multi-scale similarity metric method for multi kinds of flexible resources in VPP is proposed, which can be used to measure the similarity of different scenes in a VPP. Based on the proposed multi-scale similarity metric method, the multi-scale spectral clustering algorithm is used to obtain typical scenes of VPP. Simulation results show that based on the proposed multi-scale spectral clustering algorithm, considering different kinds of flexible resources, typical scene of VPP can be obtained effectively. And the proposed strategy has satisfying performance and high stability. Also it offers a certain guiding significance for the analysis of VPP flexibility resources, rational allocation of resources and optimization planning.

关键词： Virtual power plants Typical Scene spectral clustering algorithm Multi-scale similarity metric

来源：评论

学校读者我要写书评

暂无评论

An Improved spectral clustering algorithm Based on Dynamic Tissue-Like Membrane System 8th

An Improved Spectral Clustering Algorithm Based on Dynamic T...

引用

8th International Conference on Intelligence Science and Big Data Engineering (IScIDE)

作者： Hu, Xuewei Liu, Xiyu Shandong Normal Univ Coll Management Sci & Engn Jinan Peoples R China

ISBN: (纸本)9783030026981;9783030026974

With vast amount of data generated, it is becoming a main aspect to mine useful information from such data. clustering research is an important task of data mining. Traditional clustering algorithms such as K-means algorithm are too old to propose high-dimensional data, so an efficient clustering algorithm, spectral clustering is generated. In recent years, more and more scholars has been firmly committing to studying spectral clustering algorithm for its solid theoretical foundation and excellent clustering results. In this paper we propose an improved spectral clustering algorithm based on Dynamic Tissue-like P System abbreviated as ISC-DTP. ISC-DTP algorithm takes use of the advantages of maximal parallelism in tissue-like membrane system. Experiment is conducted on an artificial data set and four UCI data sets. And we compare the ISC-DTP algorithm with original spectral clustering algorithm and K-means algorithm. The experiments demonstrate the effectiveness and robustness of the proposed algorithm.

关键词： spectral clustering algorithm ISC-DTP algorithm Tissue-like membrane system Data mining

来源：评论

学校读者我要写书评

暂无评论

Application of spectral clustering algorithm to ES-MDA with DCT for History Matching of Gas Channel Reservoirs

引用

ENERGIES 2019年第22期12卷 4394-4394页

作者： Kim, Sungil Lee, Kyungbook Korea Inst Geosci & Mineral Resources Petr & Marine Res Div Daejeon 34132 South Korea

History matching is a calibration of reservoir models according to their production history. Although ensemble-based methods (EBMs) have been researched as promising history matching methods, reservoir parameters updated using EBMs do not have ideal geological features because of a Gaussian assumption. This study proposes an application of spectral clustering algorithm (SCA) on ensemble smoother with multiple data assimilation (ES-MDA) as a parameterization technique for non-Gaussian model parameters. The proposed method combines discrete cosine transform (DCT), SCA, and ES-MDA. After DCT is used to parameterize reservoir facies to conserve their connectivity and geometry, ES-MDA updates the coefficients of DCT. Then, SCA conducts a post-process of rock facies assignment to let the updated model have discrete values. The proposed ES-MDA with SCA and DCT gives a more trustworthy history matching performance than the preservation of facies ratio (PFR), which was utilized in previous studies. The SCA considers a trend of assimilated facies index fields, although the PFR classifies facies through a cut-off with a pre-determined facies ratio. The SCA properly decreases uncertainty of the dynamic prediction. The error rate of ES-MDA with SCA was reduced by 42% compared to the ES-MDA with PFR, although it required an extra computational cost of about 9 min for each calibration of an ensemble. Consequently, the SCA can be proposed as a reliable post-process method for ES-MDA with DCT instead of PFR.

关键词： spectral clustering algorithm ensemble smoother with multiple data assimilation discrete cosine transform preservation of facies ratio channel connectivity

来源：评论

学校读者我要写书评

暂无评论

Optimizing District Metered Areas and Associated Pressure Reducing Valves and Pumping Stations for Effective and Efficient Management of Water Distribution Network

引用

WATER RESOURCES MANAGEMENT 2025年第1期39卷 379-396页

作者： Li, Hongyan Shi, Wentao Chang, Zifeng Zhang, Feng Cui, Jianguo Lu, Xulong Mao, Libo Luliang Univ Dept Architecture & Civil Engn Luliang 033001 Peoples R China Taiyuan Univ Technol Coll Environm Sci & Engn Jinzhong 030600 Peoples R China CNNC 7 Res & Design Inst Co Ltd Inst Architectural Engn Design Taiyuan 030000 Peoples R China Shanxi Dadi Environm Investment Holdings Co Ltd Taiyuan 030000 Peoples R China

Leakage and energy consumption in urban water distribution networks (WDNs) have been receiving increasing attention, and pressure management based on district metered areas (DMAs) can effectively control leakage and save energy. In this paper, a set of scientific and practical methods for designing DMA partition of WDNs and carrying out pressure management based on it is proposed. First, the spectral clustering algorithm and NSGA-III algorithm are combined in a clever and innovative way to establish and solve the DMA boundary optimization division model. Compared with the existing methods, more factors can be considered in the process of boundary partitioning, so as to obtain overall excellent partitioning results. Second, based on the partition results, the optimal arrangement model of valves and flow meters on the DMAs boundary pipes is established and solved. Finally, pressure reducing valves (PRVs) are installed at the inlet of the DMAs, and the joint scheduling optimization model of the PRVs and pump station in WDNs is established and solved. In the process of scheduling, the pump is innovatively limited to the efficient section operation, saving energy and extending the service life of the pump. The proposed method is applied to the SH town network. Results reveal that the method can establish DMAs considering multiple objectives. Furthermore, pressure management in DMAs can not only significantly reduce leakage and energy consumption of pumping stations but also improve the water quality of networks. Therefore, this method can provide a reference for water utilities in establishing DMAs and carrying out pressure management in WDNs in the future.

关键词： Leakage District metered areas Pressure management NSGA-III spectral clustering algorithm Multicriteria decision making

来源：评论

学校读者我要写书评

暂无评论

A hybrid spectral clustering simulated annealing algorithm for the street patrol districting problem

引用

COMPLEX & INTELLIGENT SYSTEMS 2023年第2期9卷 1791-1807页

作者： Jiang, Yirui Zhao, Shan Li, Hongwei Qin, Yulu Yang, Xiaoyue Zhengzhou Univ Sch Comp & Artificial Intelligence Zhengzhou 450001 Peoples R China Zhengzhou Univ Sch Geosci & Technol Zhengzhou 450001 Peoples R China

Reasonable districting plays an important role in the patrolling process. In this paper, workload attributes are considered, and a mixed integer programming model is developed to solve the street patrol districting problem (SPDP). The improved spectral clustering algorithm named spectral clustering algorithm based on the road network (SCRn) and simulated annealing algorithm (SA) are combined. This results in a hybrid algorithm called SCRn-SA. The SCRn-SA algorithm is tested on small examples and real instances in Zhengzhou, China. The experimental results show that the proposed algorithm is effective for solving SPDP. It has better performance when compared to other advanced algorithms.

关键词： Street patrol The districting problem Road network spectral clustering algorithm Simulated annealing algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：