检索结果-内蒙古大学图书馆

Study on the country risk rating with distributed crawling system

JOURNAL OF SUPERCOMPUTING 2019年第10期75卷 6159-6177页

作者： Xie, Yuantao Wang, Wen Guo, Yabo Yang, Juan Univ Int Business & Econ Sch Insurance & Econ Beijing Peoples R China China Export & Credit Insurance Corp Beijing Peoples R China Chinese Acad Sci & Technol Dev Inst Comprehens Dev Beijing Peoples R China

This paper built a system containing the distributed crawling Module, the Database Module and the Analysis Module to collect a large number of objective data, clean the data and realize the business intelligence representation. The distributed crawling system includes the distributed crawling Module building from Hadoop, the Database Module by SQL Server and the Analysis Module by the SAS system. The first two modules support a distributed way to collect and convert non-structural data into structural data for the last module processing. Then, this paper extends research fields from theory to application about B&R, constructs rating system from the perspective of political risk, economic risk, financial risk, business environment risk and legal risk, establishes a 140 rating index, collects 46,200 sample data, and adopts Model of Principal Components Analysis, Analytic Hierarchy Process and Efficacy Function to access "the Belt & Road Initiative" 66 countries for 5 consecutive years of export credit insurance in the country risk rating. This paper also gives a detail explanation on 2015 rating results, showing that, Singapore wins the highest credit rating among the country;the credit rating of Latvia, Estonia, Slovakia, Turkey, Malaysia, Russia, Thailand and other countries is very high;Afghanistan, Ukraine, Laos, Iran, Arabia, Republic of Syria, Iraq, Burma, Republic of Yemen, East Timor and other countries with poor credit ratings. The conclusion is consistent with the domestic and overseas well-known rating agencies.

关键词： The Belt and Road Initiative Export credit insurance The country risk distributed crawling Hadoop

来源：评论

学校读者我要写书评

暂无评论

A GNP-based Scheduling Strategy for distributed crawling

A GNP-based Scheduling Strategy for Distributed Crawling

引用

International Conference on Web Information Systems and Mining

作者： Liu, Shuang Xu, Xiao Li, Dong Zhang, Wei-zhe Liu, Xin-ran Harbin Inst Technol Dept Comp Sci & Technol Harbin 150006 Peoples R China Coordinat Ctr China Natl Comp Network Emergency Response Tech Team Beijing Peoples R China

ISBN: (纸本)9780769538174

In order to solve task scheduling and load balancing problems of distributed search engines, a GNP-based scheduling strategy for distributed crawling and a load balancing method are proposed in this paper. Internet distance estimating mechanism is adopted as a replacement for large-scale network distance measurement, which not only improves response speed of the system, but also reduces loads on WAN caused by the system. Through deploying crawling nodes at WANs, we built a distributed search engine, and implemented several scheduling strategies. The online experiment shows great improvement in system's performance.

关键词： distributed crawling scheduling strategies load balancing network measurement GNP

来源：评论

学校读者我要写书评

暂无评论

IPMicra: An IP-address based location aware distributed web crawler

IPMicra: An IP-address based location aware distributed web ...

引用

International Conference on Internet Computing/International Symposium on Web Services and Applications

作者： Papapetrou, O Samaras, G Univ Cyprus Dept Comp Sci Nicosia Cyprus

ISBN: (纸本)1932415467

distributed crawling is able to overcome important limitations of the traditional single-sourced web crawling systems. However, the optimal benefit of distributed crawling is usually limited to the sites hosting the crawlers, the rest of the URLs are by large randomly distributed to the various crawlers. In this work, we propose a location-aware method, called IPMicra, that utilizes an IP address hierarchy, and allows crawling of links in a near optimal location aware manner Our proposal outperforms earlier distributed crawling schemes by requiring one order of magnitude less time for crawling of the same set of sites.

关键词： distributed crawling web crawling location aware crawling

来源：评论

学校读者我要写书评

暂无评论

Application of VM-Based Computations to Speedup the Web crawling Process on Multi-Core Processors

Application of VM-Based Computations to Speedup the Web Craw...

引用

12th International Symposium on distributed Computing and Applications to Business, Engineering and Science (DCABES)

作者： Al-Bahadili, Hussein Qtishat, Hamzah Univ Petra Fac Informat Technol Amman Jordan Middle East Univ Fac Inf Technol Amman Jordan

ISBN: (纸本)9780769550602

A Web crawler is an important component of the Web search engine. It demands large amount of hardware resources to crawl data from the rapidly growing and changing Web. The crawling process should be performed continuously to maintain up-to-date data. This paper develops a new approach to speed up the crawling process on a multi-core processor by utilizing the concept of virtualization. In this approach, the multi-core processor is divided into a number of virtual-machines (VMs), which can concurrently perform different crawling tasks on different initial data. It presents a description, implementation, and evaluation of a VM-based distributed Web crawler. The speedup factor achieved by the VM-based crawler over no virtualization crawler, for crawling various numbers of documents, is estimated. Also, the effect of number of VMs on the speedup factor is investigated.

关键词： Web search engine Web crawler virtualization virtual machines distributed crawling multi-core processor processor-farm methodology

来源：评论

学校读者我要写书评

暂无评论

Application of VM-Based Computations to Speedup the Web crawling Process on Multi-Core Processors

Application of VM-Based Computations to Speedup the Web Craw...

引用

第12届分布式计算及其应用国际学术研讨会

作者： Hussein Al-Bahadili Hamzah Qtishat Faculty of Information Technology University of Petra Amman Jordan

A Web crawler is an important component of the Web search engine. It demands large amount of hardware resources to crawl data from the rapidly growing and changing Web. The crawling process should be performed continuously to maintain up-to-date data. This paper develops a new approach to speed up the crawling process on a multi-core processor by utilizing the concept of virtualization. In this approach, the multi-core processor is divided into a number of virtual-machines(VMs), which can concurrently perform different crawling tasks on different initial data. It presents a description, implementation, and evaluation of a VM-based distributed Web crawler. The speedup factor achieved by the VM-based crawler over no virtualization crawler, for crawling various numbers of documents, is estimated. Also, the effect of number of VMs on the speedup factor is investigated.

关键词： Web search engine Web crawler virtualization virtual machines distributed crawling multi-core processor processor-farm methodology.

来源：评论

学校读者我要写书评

暂无评论

Application of VM-Based Computations to Speedup the Web crawling Process on Multi-Core Processors

Application of VM-Based Computations to Speedup the Web Craw...

引用

The 12th International Symposium on distributed Computing and Applications to Business,Engineering and Science(DCABES 2013)(第十二届分布式计算及其应用国际学术研讨会)

作者： Hussein Al-Bahadili Hamzah Qtishat Faculty of Information Technology University of Petra Amman Jordan Faculty of Information Technology Middle East University Amman Jordan

A Web crawler is an important component of the Web search *** demands large amount of hardware resources to crawl data from the rapidly growing and changing *** crawling process should be performed continuously to maintain up-to-date *** paper develops a new approach to speed up the crawling process on a multi-core processor by utilizing the concept of *** this approach,the multi-core processor is divided into a number of virtual-machines(VMs),which can concurrently perform different crawling tasks on different initial *** presents a description,implementation,and evaluation of a VM-based distributed Web *** speedup factor achieved by the VM-based crawler over no virtualization crawler,for crawling various numbers of documents,is ***,the effect of number of VMs on the speedup factor is investigated.

关键词： Web search engine Web crawler virtualization virtual machines distributed crawling multi-core processor processor-farm methodology

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：