检索结果-内蒙古大学图书馆

作者： Fejfar, Petr Charles University of Prague

Title: Interactive crawling and data extraction Author: Bc. Petr Fejfar Authors e-mail address: pfejfar@*** Department: Department of Distributed and Dependable Systems Supervisor: Mgr. Pavel Je ek, Ph.D., Department of Distributed and De- pendable Systems Abstract: The subject of this thesis is web crawling and data extraction from Rich Internet Applications (RIA). The thesis starts with analysis of modern web pages along with techniques used for crawling and data extraction. Based on this analysis, we designed a tool which crawls RIAs according to the instructions defined by the user via graphic interface. In contrast with other currently popular tools for RIAs, our solution is targeted at users with no programming experience, including business and analyst users. The designed solution itself is implemented in form of RIA, using the web- Driver protocol to automate multiple browsers according to user-defined instructions. Our tool allows the user to inspect browser sessions by dis- playing pages that are being crawled simultaneously. This feature enables the user to troubleshoot the crawlers. The outcome of this thesis is a fully design and implemented tool enabling business user to extract data from the RIAs. This opens new opportunities for this type of user to collect data from web pages for use...

关键词： web crawling web data extraction web scraping AJAX RIA Rich Internet Application browser automation

来源：评论

学校读者我要写书评

暂无评论

FSA and NLP based Un-supervised non template web data extraction in the construction Of Dynamic Ontology 16

FSA and NLP based Un-supervised non template Web data extrac...

引用

1st International Conference on Informatics and Analytics (ICIA)

作者： Kumar, Uma Pavan K. Saraswathi, S. Pondicherry Engn Coll Dept Comp Sci & Engn Pondicherry India Pondicherry Engn Coll Dept Informat Technol Pondicherry India

ISBN: (纸本)9781450347563

The main focus of our research work is to construct the Job Recommendation System (JRS) based on the ontology construction. The current discussion is related to web data extraction process in the construction of *** importance of this concept is how best data can be extracted from various web pages to minimize the time requirement and improve the efficiency. The process followed to construct ontology is first identified various sources of job portals, then extraction of data from those portals for that the proposed a method is FSA(Finite State Automata) and NLP(Natural Language Processing) based extraction of data. The outcome of this method is efficient data extraction in terms of time and space usage when compared with other models.

关键词： Ontology web data extraction Job Recommendation System (JRS) FSA NLP Dynamic Ontology Un supervised learning

来源：评论

学校读者我要写书评

暂无评论

Practical web data extraction Are we there yet? - A Short Survey

Practical Web Data Extraction Are we there yet? - A Short Su...

引用

IEEE/WIC/ACM International Conference on web Intelligence (WI)

作者： Schulz, Andreas Laessig, Joerg Gaedke, Martin Univ Appl Sci Zittau Gorlitz Dept Comp Sci Gorlitz Germany Tech Univ Chemnitz Dept Comp Sci Chemnitz Germany

ISBN: (纸本)9781509044702

The number of web documents as well as the inherent data and information is growing at a rapid pace. The interest in extracting and utilizing this data is rising likewise. The prospects that are unlocked by web data extraction to its users are as broad as the extensiveness of topics and fields on the web. The major obstacle is to utilize the available data, contents and processes. Several, mostly older survey papers have already shown developments and approaches to solve web data extraction tasks, but there is a need for a more up-to-date review, showing the latest developments. Additionally when looking from the user perspective, there is still a gap between research results and practical applicability. Available solutions, including research results, commercial products and open source solutions lack certain capabilities or suffer from severe usability issues. This paper therefore gives a short review of the state of the art in web data extraction and relates this to the practical application of these technologies.

关键词： web data extraction web Mining Wrapper Induction

来源：评论

学校读者我要写书评

暂无评论

web data extraction Based On Visual Information and Partial Tree Alignment 11

Web Data Extraction Based On Visual Information and Partial ...

引用

11th web Information System and Application Conference

作者： Fan, Siwu Wang, Xinjun Dong, Yongquan Shandong Univ Sch Comp Sci & Technol Jinan Peoples R China

ISBN: (纸本)9781479957279

web databases contain a huge amount of structured data which are easily obtained via their query interfaces only. The query results are presented in dynamically generated web pages, usually in the form of data records, for human use. The automatical web data extraction is critical in web integration. A number of approaches have been proposed. The early work are most based on the source code or the tag tree of the page. Recent approaches use the visual feature to extract data information, which are better than the previous work. However, these approaches still have inherent limitation. In this paper, we propose a novel approach that make use of visual features to extract data information from web page, including the data records and the data items. The results of this experiment tests on a large set of query result pages in different domain show that the proposed approach is highly effective.

关键词： web data extraction Wrapper generation web mining

来源：评论

学校读者我要写书评

暂无评论

Extensible Markup Language web data retrieval using hybrid deep learning model

引用

APPLIED SOFT COMPUTING 2025年 175卷

作者： Gopianand, M. Muthumeenakshi, R. PSNA Coll Engn & Technol Dept Comp Applicat Dindigul 624622 Tamil Nadu India REVA Univ Sch Comp Sci & Applicat Bengaluru India

Objectives: An Extensible Markup Language (XML) document is a kind of text file that utilizes the markup language to store data in an easily readable format for both machines and humans. In the data retrieval process, the client will send a request to the database or web server, which then retrieves the requested data and returns it in XML format. However, XML documents are complex and intricate to navigate, also it is difficult to identify and access the particular data attribute or element. Therefore, the primary objective of this research is to develop a novel Deep Learning (DL) model for XML web data retrieval. Methods: This research proposed the Deep Fuzzy Clustering Bidirectional Gated Random Multimodel Deep Learning (DFC-BiGRMDL) for efficient data extraction and refinement. At first, the XML web document is collected from a specific database. Then, the user query and XML web data are employed to check similarity by using a new search algorithm that combines hybrid semantic similarity measures, which are combined using Mahalanobis distance and Clark similarity measures. It enables the identification of relevant data based on the user's queries. The Deep Fuzzy Clustering (DFC) model is used to group the retrieved XML web data. At last, refinement from retrieved XML web data is performed using the proposed DFC-BiGRMDL model, which is designed by the integration of Bidirectional Gated Recurrent Unit (BiGRU) and Random Multimodel Deep Learning (RMDL). Finding: The performance of the proposed DFC-BiGRMDL model is evaluated using metrics, like Clark similarity, precision, recall and F-measure that have reached maximum values of 0.908, 91.360 %, 92.367 % and 91.860 % respectively. Improvement: The developed model enhances the data extraction and refinement by combining advanced techniques like Clark similarity measures and Mahalanobis distance, and also, DFC for more accurate data retrieval. Furthermore, the hybridization of BiGRU and RMDL improves the learning from

关键词： XML document Clustering web data extraction Deep Fuzzy Clustering Bidirectional Gated Recurrent Units

来源：评论

学校读者我要写书评

暂无评论

Automatic Regular Expression Generation for Extracting Relevant Image data From web Pages Using Genetic Algorithms

引用

IEEE ACCESS 2024年 12卷 90660-90669页

作者： Aslanyurek, Canan Yerlikaya, Tarik Kirklareli Univ TR-39100 Kirklareli Turkiye Trakya Univ TR-22030 Edirne Turkiye

In this study, a method that automatically generates regular expressions using genetic algorithms is designed to extract relevant images on web pages. data extraction, which is usually done with web scrapers, can also be done with regular expressions. The complexity of regular expressions and the fact that they require expert knowledge make their writing difficult. With this study, a regular expression is automatically created to obtain relevant images of news content on websites. With the principle of genetic algorithms, the survival of the good and the elimination of the bad, a regular expression that can reach the most relevant image is produced. Thus, instead of a time-consuming and error-prone method such as creating the appropriate pattern for each site with web scraper tools, automatic regular expression generation using genetic algorithm methods can be used as a better method. A data set containing text-based related and irrelevant images from 200 websites collected from 58 countries was used in the study. There are 22,682 relevant images among 635,015 image data in the dataset. With the method developed using the genetic algorithm, the rate of accessing the relevant images by regular expressions produced by only looking at the relevant image data is approximately 98.49%.

关键词： data mining Genetic algorithms web pages Feature extraction Statistics Sociology Manuals Image reconstruction automatic regular expressions web data extraction image data extraction

来源：评论

学校读者我要写书评

暂无评论

Trinity: On Using Trinary Trees for Unsupervised web data extraction

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND data ENGINEERING 2014年第6期26卷 1544-1556页

作者： Sleiman, Hassan A. Corchuelo, Rafael Univ Seville ETSI Informat E-41012 Seville Spain

web data extractors are used to extract data from web documents in order to feed automated processes. In this article, we propose a technique that works on two or more web documents generated by the same server-side template and learns a regular expression that models it and can later be used to extract data from similar documents. The technique builds on the hypothesis that the template introduces some shared patterns that do not provide any relevant data and can thus be ignored. We have evaluated and compared our technique to others in the literature on a large collection of web documents;our results demonstrate that our proposal performs better than the others and that input errors do not have a negative impact on its effectiveness;furthermore, its efficiency can be easily boosted by means of a couple of parameters, without sacrificing its effectiveness.

关键词： web data extraction automatic wrapper generation wrappers unsupervised learning

来源：评论

学校读者我要写书评

暂无评论

Aesthetic Trends and Semantic web Adoption of Media Outlets Identified through Automated Archival data extraction

引用

FUTURE INTERNET 2022年第7期14卷 204-204页

作者： Lamprogeorgos, Aristeidis Pergantis, Minas Panagopoulos, Michail Giannakoulopoulos, Andreas Ionian Univ Dept Audio & Visual Arts 7 Tsirigoti Sq Corfu 49100 Greece

The last decade has been a time of great progress in the World Wide web and this progress has manifested in multiple ways, including both the diffusion and expansion of Semantic web technologies and the advancement of the aesthetics and usability of web user interfaces. Online media outlets have often been popular web destinations and so they are expected to be at the forefront of innovation, both in terms of the integration of new technologies and in terms of the evolution of their interfaces. In this study, various web data extraction techniques were employed to collect current and archival data from news websites that are popular in Greece, in order to monitor and record their progress through time. This collected information, which took the form of a website's source code and an impression of their homepage in different time instances of the last decade, has been used to identify trends concerning Semantic web integration, DOM structure complexity, number of graphics, color usage, and more. The identified trends were analyzed and discussed with the purpose of gaining a better understanding of the ever-changing presence of the media industry on the web. The study concluded that the introduction of Semantic web technologies in online media outlets was rapid and extensive and that website structural and visual complexity presented a steady and significant positive trend, accompanied by increased adherence to color harmony.

关键词： media industry online media news websites web data extraction Semantic web web aesthetics web archives World Wide web

来源：评论

学校读者我要写书评

暂无评论

web data extraction Based On Visual Information and Partial Tree Alignment

Web Data Extraction Based On Visual Information and Partial ...

引用

2014 11th web Information System and Application Conference(第十一届全国web信息系统及其应用学术会议)

作者： Siwu Fan Xinjun Wang Yongquan Dong School of Computer Science and Technology Shandong University Jinan China School of Computer Science and Technology Jiangsu Normal University Xuzhou China

ISBN: (纸本)9781479957286

web databases contain a huge amount of structured data which are easily obtained via their query interfaces *** query results are presented in dynamically generated web pages, usually in the form of data records, for human *** automatical web data extraction is critical in web integration.A number of approaches have been *** early work are most based on the source code or the tag tree of the *** approaches use the visual feature to extract data information, which are better than the previous ***, these approaches still have inherent *** this paper, we propose a novel approach that make use of visual features to extract data information from web page,including the data records and the data *** results of this experiment tests on a large set of query result pages in different domain show that the proposed approach is highly effective.

关键词： web data extraction Wrapper generation web mining

来源：评论

学校读者我要写书评

暂无评论

Scraping Relevant Images fromweb Pages without Download

引用

ACM TRANSACTIONS ON THE web 2024年第1期18卷 1-27页

作者： Uzun, Erdinc Tekirdag Namik Kemal Univ Comp Engn Dept Corlu Fac Engn TR-59860 Corlu Tekirdag Turkiye

Automatically scraping relevant images from web pages is an error-prone and time-consuming task, leading experts to prefer manually preparing extraction patterns for a website. Existing web scraping tools are built on these patterns. However, this manual approach is laborious and requires specialized knowledge. Automatic extraction approaches, while a potential solution, require large training datasets and numerous features, including width, height, pixels, and file size, that can be difficult and time-consuming to obtain. To address these challenges, we propose a semi-automatic approach that does not require an expert, utilizes small training datasets, and has a low error rate while saving time and storage. Our approach involves clustering web pages from a website and suggesting several pages for a non-expert to annotate relevant images. The approach then uses these annotations to construct a learning model based on textual data from the HTML elements. In the experiments, we used a dataset of 635,015 images from 200 news websites, each containing 100 pages, with 22,632 relevant images. When comparing several machine learning methods for both automatic approaches and our proposed approach, the AdaBoost method yields the best performance results. When using automatic extraction approaches, the best f-Measure that can be achieved is 0.805 with a learning model constructed from a large training dataset consisting of 120 websites (12,000 web pages). In contrast, our approach achieved an average f-Measure of 0.958 for 200 websites with only six web pages annotated per website. This means that a non-expert only needs to examine 1,200 web pages to determine the relevant images for 200 websites. Our approach also saves time and storage space by not requiring the download of images and can be easily integrated into currently available web scraping tools, because it is based on textual data.

关键词： web mining relevant images web data extraction crawler design and evaluation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：