检索结果-内蒙古大学图书馆

SNCD: A fast and scalable distributed near-miss code clone detector for big code based on partial index

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 167卷

作者： Yang, Liming Zhao, Jun Xie, Rulin Ren, Yi Guan, Jianbo Li, Bao Ma, Jun Tan, Yusong Natl Univ Def Technol Coll Comp Sci & Technol 137 Yanwachi St Changsha 410073 Hunan Peoples R China

A number of techniques have been proposed over the years to detect clones for improving software maintenance, reusability or security. However, there is still a lack of language agnostic approaches with code granularity flexibility for near-miss clone detection in big code in scale. It is challenging to detect near-miss clones in big code across large scale source repositories with hundreds of millions of lines of code (MLOC) or more. The main reason is that it requires more computing and memory resources as the scale of the source code increases. In particular, near-miss clone detection is more difficult and need more resources. In this paper, we present SNCD, a fast and scalable distributed clone detection approach. It overcomes single node CPU and memory resource limitation with MapReduce and HDFS by scalable distributed parallelization. Furthermore, it is partial index based and optimized with multi-threading strategy which further improve the efficiency. It can not only detect Type-1 and Type-2 clones but can also discover the most computationally expensive Type-3 clones for large repositories. Meanwhile, it works for both function and file granularities, and it supports many different programming languages. Experimental results show that SNCD scales better for big code with the size of code in terms of lines of code increases compared to existing clone detection techniques, with recall and precision comparable to state-of-art approaches. With bigCloneBench and the Mutation Framework, two recent and widely used benchmarks, SNCD achieves both high recall and precision, which is competitive with other existing tools.

关键词： Near-miss code clone detection big code Distributed parallelization Partial index Efficiency and scalability

来源：评论

学校读者我要写书评

暂无评论

Scalable Taint Specification Inference with big code 2019

Scalable Taint Specification Inference with Big Code

引用

40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) part of ACM's Federated Computing Research Conference (FCRC)

作者： Chibotaru, Victor Bichsel, Benjamin Raychev, Veselin Vechev, Martin DeepCode AG Zurich Switzerland Swiss Fed Inst Technol Zurich Switzerland

ISBN: (纸本)9781450367127

We present a new scalable, semi-supervised method for inferring taint analysis specifications by learning from a large dataset of programs. Taint specifications capture the role of library APIs (source, sink, sanitizer) and are a critical ingredient of any taint analyzer that aims to detect security violations based on information flow. The core idea of our method is to formulate the taint specification learning problem as a linear optimization task over a large set of information flow constraints. The resulting constraint system can then be efficiently solved with stateof-the-art solvers. Thanks to its scalability, our method can infer many new and interesting taint specifications by simultaneously learning from a large dataset of programs (e.g., as found on GitHub), while requiring few manual annotations. We implemented our method in an end-to-end system, called Seldon, targeting Python, a language where static specification inference is particularly hard due to lack of typing information. We show that Seldon is practically effective: it learned almost 7, 000 API roles from over 210, 000 candidate APIs with very little supervision (less than 300 annotations) and with high estimated precision (67%). Further, using the learned specifications, our taint analyzer flagged more than 20, 000 violations in open source projects, 97% of which were undetectable without the inferred specifications.

关键词： Specification Inference Taint Analysis big code

来源：评论

学校读者我要写书评

暂无评论

Predicting Program Properties from "big code" 15

Predicting Program Properties from "Big Code"

引用

42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 15)

作者： Raychev, Veselin Vechev, Martin Krause, Andreas Swiss Fed Inst Technol Dept Comp Sci Zurich Switzerland

ISBN: (纸本)9781450333009

We present a new approach for predicting program properties from massive codebases (aka "big code"). Our approach first learns a probabilistic model from existing data and then uses this model to predict properties of new, unseen programs. The key idea of our work is to transform the input program into a representation which allows us to phrase the problem of inferring program properties as structured prediction in machine learning. This formulation enables us to leverage powerful probabilistic graphical models such as conditional random fields (CRFs) in order to perform joint prediction of program properties. As an example of our approach, we built a scalable prediction engine called JSNICE(1) for solving two kinds of problems in the context of JavaScript: predicting (syntactic) names of identifiers and predicting (semantic) type annotations of variables. Experimentally, JSNICE predicts correct names for 63% of name identifiers and its type annotation predictions are correct in 81% of the cases. In the first week since its release, JSNICE was used by more than 30, 000 developers and in only few months has become a popular tool in the JavaScript developer community. By formulating the problem of inferring program properties as structured prediction and showing how to perform both learning and inference in this context, our work opens up new possibilities for attacking a wide range of difficult problems in the context of "big code" including invariant generation, de-compilation, synthesis and others.

关键词： names big code structured prediction javascript program properties conditional random fields closure compiler types

来源：评论

学校读者我要写书评

暂无评论

A Survey of Machine Learning for big code and Naturalness

引用

ACM COMPUTING SURVEYS 2018年第4期51卷 81-81页

作者： Allamanis, Miltiadis Barr, Earl T. Devanbu, Premkumar Sutton, Charles Microsoft Res 21 Stn Rd Cambridge CB1 2FB England UCL Dept Comp Sci Gower St London WC1E 6BT England Univ Calif Davis Dept Comp Sci Davis CA 95616 USA Univ Edinburgh Sch Informat Edinburgh EH8 9AB Midlothian Scotland Alan Turing Inst London England

Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of source code that exploit the abundance of patterns of code. In this article, we survey this work. We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models. We present a taxonomy based on the underlying design principles of eachmodel and use it to navigate the literature. Then, we review how researchers have adapted these models to application areas and discuss cross-cutting and application-specific challenges and opportunities.

关键词： big code code naturalness software engineering tools machine learning

来源：评论

学校读者我要写书评

暂无评论

IBFET: Index-based features extraction technique for scalable code clone detection at file level granularity

引用

SOFTWARE-PRACTICE & EXPERIENCE 2020年第1期50卷 22-46页

作者： Akram, Junaid Mumtaz, Majid Luo, Ping Tsinghua Univ Sch Software Key Lab Informat Syst Secur Beijing Peoples R China

Many techniques have been developed over the years to detect code clones in different software systems to maintain security measures. These techniques often require the source code to compare the subject system against a very large data set of big code. This paper presents index-based features extraction technique (IBFET) to detect code clones at a very large-scale level to billions of LOC at file level granularity. We performed preprocessing, indexing, and clone detection for more than 324 billion of LOC using a Hadoop distributed environment, which is quite faster and more efficient as compared to existing distributed indexing and clone detection techniques;meanwhile, it detects all three types of clones efficiently. The MapReduce rule of divide and conquer is used for a count and retrieve the similar features between different systems. We evaluated the execution time, scalability, precision, and recall of IBFET by using a well-known clone detection data set IJaDataset and bigCloneBench;furthermore, we compared the results with other state-of-the-art tools. Our approach is faster, flexible, scalable, and provides accurate results with high authenticity and can be implemented at a large-scale level.

关键词： big code clone detection code similarity detection plagiarism detection software reuse software security and maintenance

来源：评论

学校读者我要写书评

暂无评论

Deep Learning for Source code Modeling and Generation: Models, Applications, and Challenges

引用

ACM COMPUTING SURVEYS 2020年第3期53卷 62-62页

作者： Le, Triet H. M. Chen, Hao Babar, Muhammad Ali Univ Adelaide Sch Comp Sci Adelaide SA 5005 Australia

Deep Learning (DL) techniques for Natural Language Processing have been evolving remarkably fast. Recently, the DL advances in language modeling, machine translation, and paragraph understanding are so prominent that the potential of DL in Software Engineering cannot be overlooked, especially in the field of program learning. To facilitate further research and applications of DL in this field, we provide a comprehensive review to categorize and investigate existing DL methods for source code modeling and generation. To address the limitations of the traditional source code models, we formulate common program learning tasks under an encoder-decoder framework. After that, we introduce recent DL mechanisms suitable to solve such problems. Then, we present the state-of-the-art practices and discuss their challenges with some recommendations for practitioners and researchers as well.

关键词： Deep learning big code source code modeling source code generation

来源：评论

学校读者我要写书评

暂无评论

SQVDT: A scalable quantitative vulnerability detection technique for source code security assessment

引用

SOFTWARE-PRACTICE & EXPERIENCE 2021年第2期51卷 294-318页

作者： Akram, Junaid Luo, Ping Tsinghua Univ Sch Software Key Lab Informat Syst Secur Beijing Peoples R China

Vulnerability detection and exploit is becoming a very important part of security, especially in malware code delivery, hacking a system, efforts to create patches, improving the source code, or updating a software. Vulnerabilities in applications, including browsers, media players, online services, document readers, and so forth. are often exploited and cause a serious damage. In this article, we propose a vulnerability detection technique to detect vulnerabilities in software, as well as shared libraries at source code level. We crawl the vulnerable source code by tracing and locating the patch files from different web sources according to their CVE-numbers and built a fingerprint index of 2931 vulnerable files. Then we developed a vulnerability detection approach based on code clone detection technique and detect hundreds of vulnerabilities in thousands of GitHub open source projects, which are not noticed before as vulnerable. We detected vulnerabilities in some very famous recently available software, including latest version of Linux, HTC-kernel, FindX-8.1-kernel, and in 7-TB of C/C++ source code (152,823 open source projects). In this study, we discuss some of the very high severity level (CVSS) vulnerabilities that are detected by our approach. Furthermore, we performed an empirical evaluation and verification on these vulnerabilities, including intraproject clone vulnerabilities, copied-kernel clone vulnerabilities, and library-used clone vulnerabilities. Our technique is very fast, efficient, reliable, practical, scalable, and can be implemented at industrial level. The comparison with the state-of-the-art tools shows the effectiveness of our approach.

关键词： big code clone detection code vulnerability detection software reuse software security and maintenance

来源：评论

学校读者我要写书评

暂无评论

Learning the Relation Between code Features and code Transforms With Structured Prediction

引用

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 2023年第7期49卷 3872-3900页

作者： Yu, Zhongxing Martinez, Matias Chen, Zimin Bissyande, Tegawende F. F. Monperrus, Martin Shandong Univ Jinan Peoples R China Univ Politecn Cataluna Barcelona Spain KTH Royal Inst Technol S-11428 Stockholm Sweden Univ Luxembourg L-4365 Esch Sur Alzette Luxembourg

To effectively guide the exploration of the code transform space for automated code evolution techniques, we present in this article the first approach for structurally predicting code transforms at the level of AST nodes using conditional random fields (CRFs). Our approach first learns offline a probabilistic model that captures how certain code transforms are applied to certain AST nodes, and then uses the learned model to predict transforms for arbitrary new, unseen code snippets. Our approach involves a novel representation of both programs and code transforms. Specifically, we introduce the formal framework for defining the so-called AST-level code transforms and we demonstrate how the CRF model can be accordingly designed, learned, and used for prediction. We instantiate our approach in the context of repair transform prediction for Java programs. Our instantiation contains a set of carefully designed code features, deals with the training data imbalance issue, and comprises transform constraints that are specific to code. We conduct a large-scale experimental evaluation based on a dataset of bug fixing commits from real-world Java projects. The results show that when the popular evaluation metric top-3 is used, our approach predicts the code transforms with an accuracy varying from 41% to 53% depending on the transforms. Our model outperforms two baselines based on history probability and neural machine translation (NMT), suggesting the importance of considering code structure in achieving good prediction accuracy. In addition, a proof-of-concept synthesizer is implemented to concretize some repair transforms to get the final patches. The evaluation of the synthesizer on the Defects4j benchmark confirms the usefulness of the predicted AST-level repair transforms in producing high-quality patches.

关键词： code transform big code machine learning program repair

来源：评论

学校读者我要写书评

暂无评论

code2vec: Learning Distributed Representations of code

引用

PROCEEDINGS OF THE ACM ON PROGRAMMING LANGUAGES-PACMPL 2019年第POPL期3卷 1-29页

作者： Alon, Uri Zilberstein, Meital Levy, Omer Yahav, Eran Technion Haifa Israel Facebook AI Res Seattle WA USA

We present a neural model for representing snippets of code as continuous distributed vectors ("code embeddings"). The main idea is to represent a code snippet as a single fixed-length code vector, which can be used to predict semantic properties of the snippet. To this end, code is first decomposed to a collection of paths in its abstract syntax tree. Then, the network learns the atomic representation of each path while simultaneously learning how to aggregate a set of them. We demonstrate the effectiveness of our approach by using it to predict a method's name from the vector representation of its body. We evaluate our approach by training a model on a dataset of 12M methods. We show that code vectors trained on this dataset can predict method names from files that were unobserved during training. Furthermore, we show that our model learns useful method name vectors that capture semantic similarities, combinations, and analogies. A comparison of our approach to previous techniques over the same dataset shows an improvement of more than 75%, making it the first to successfully predict method names based on a large, cross-project corpus. Our trained model. visualizations and vector similarities are available as an interactive online demo at http://***. The code, data and trained models are available at https://***/tech-srl/code2vec.

关键词： big code Machine Learning Distributed Representations

来源：评论

学校读者我要写书评

暂无评论

MUSYNTH: Program Synthesis via code Reuse and code Manipulation 9th

MUSYNTH: Program Synthesis via Code Reuse and Code Manipulat...

引用

9th International Symposium on Search Based Software Engineering (SSBSE)

作者： Kashyap, Vineeth Swords, Rebecca Schulte, Eric Melski, David GrammaTech Inc Ithaca NY 14850 USA

ISBN: (纸本)9783319662992;9783319662985

MUSYNTH takes a draft C program with "holes", a test suite, and optional simple hints-that together specify a desired functionality-and performs program synthesis to auto-complete the holes. First, MUSYNTH leverages a similar-code-search engine to find potential "donor" code (similar to the required functionality) from a corpus. Second, MuSynth applies various synthesis mutations in an evolutionary loop to find and modify the donor code snippets to fit the input context and produce the expected functionality. This paper focuses on the latter, and our preliminary evaluation shows that MUSYNTH's combination of type-based heuristics, simple hints, and evolutionary search are each useful for efficient program synthesis.

关键词： Program synthesis Evolutionary computation code reuse big code

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：