Hunting binary code difference without source code (i.e., binary diffing) has compelling applications in software security. Due to the high variability of binarycode, existing solutions have been driven towards measu...
详细信息
ISBN:
(纸本)9781450383912
Hunting binary code difference without source code (i.e., binary diffing) has compelling applications in software security. Due to the high variability of binarycode, existing solutions have been driven towards measuring semantic similarities from syntactically different code. Since compiler optimization is the most common source contributing to binary code differences in syntax, testing the resilience against the changes caused by different compiler optimization settings has become a standard evaluation step for most binary diffing approaches. For example, 47 top-venue papers in the last 12 years compared different program versions compiled by default optimization levels (e.g., -Ox in GCC and LLVM). Although many of them claim they are immune to compiler transformations, it is yet unclear about their resistance to non-default optimization settings. Especially, we have observed that adversaries explored non-default compiler settings to amplify malware differences. This paper takes the first step to systematically studying the effectiveness of compiler optimization on binary code differences. We tailor search-based iterative compilation for the auto-tuning of binary code differences. We develop BinTuner to search near-optimal optimization sequences that can maximize the amount of binary code differences. We run BinTuner with GCC 10.2 and LLVM 11.0 on SPEC benchmarks (CPU2006 & CPU2017), Coreutils, and OpenSSL. Our experiments show that at the cost of 279 to 1, 881 compilation iterations, BinTuner can find custom optimization sequences that are substantially better than the general -Ox settings. BinTuner's outputs seriously undermine prominent binary diffing tools' comparisons. In addition, the detection rate of the IoT malware variants tuned by BinTuner falls by more than 50%. Our findings paint a cautionary tale for security analysts that attackers have a new way to mutate malware code cost-effectively, and the research community needs to step back to reassess opti
binary analysis is an important capability required for many security and software engineering applications. Consequently, there are many binary analysis techniques and tools with varied capabilities. However, testing...
详细信息
ISBN:
(纸本)9781450394758
binary analysis is an important capability required for many security and software engineering applications. Consequently, there are many binary analysis techniques and tools with varied capabilities. However, testing these tools requires a large, varied binary dataset with corresponding source-level information. In this paper, we present CORNUCOPIA, an architecture agnostic automated framework that can generate a plethora of binaries from corresponding program source by exploiting compiler optimizations and feedbackguided learning. Our evaluation shows that CORNUCOPIA was able to generate 309K binaries across four architectures (x86, x64, ARM, MIPS) with an average of 403 binaries for each program and outperforms BINTUNER [53], a similar technique. Our experiments revealed issues with the LLVM optimization scheduler resulting in compiler crashes (similar to 300). Our evaluation of four popular binary analysis tools ANGR, GHIDRA, IDA, and RADARE, using CORNUCOPIA generated binaries, revealed various issues with these tools. Specifically, we found 263 crashes in angr and one memory corruption issue in ida. Our differential testing on the analysis results revealed various semantic bugs in these tools. We also tested machine learning tools, ASM2VEC, SAFE, and DEBIN, that claim to capture binary semantics and show that they perform poorly (e.g., DEBIN F1 score dropped to 12.9% from reported 63.1%) on CORNUCOPIA generated binaries. In summary, our exhaustive evaluation shows that CORNUCOPIA is an effective mechanism to generate binaries for testing binary analysis techniques effectively.
binary analysis is an important capability required for many security and software engineering applications. Consequently, there are many binary analysis techniques and tools with varied capabilities. However, testing...
详细信息
ISBN:
(纸本)9781450394758
binary analysis is an important capability required for many security and software engineering applications. Consequently, there are many binary analysis techniques and tools with varied capabilities. However, testing these tools requires a large, varied binary dataset with corresponding source-level information. In this paper, we present Cornucopia, an architecture agnostic automated framework that can generate a plethora of binaries from corresponding program source by exploiting compiler optimizations and feedback-guided learning. Our evaluation shows that Cornucopia was able to generate 309K binaries across four architectures (x86, x64, ARM, MIPS) with an average of 403 binaries for each program and outperforms BinTuner [53], a similar technique. Our experiments revealed issues with the LLVM optimization scheduler resulting in compiler crashes (∼ 300). Our evaluation of four popular binary analysis tools angr, Ghidra, ida, and radare, using Cornucopia generated binaries, revealed various issues with these tools. Specifically, we found 263 crashes in angr and one memory corruption issue in ida. Our differential testing on the analysis results revealed various semantic bugs in these tools. We also tested machine learning tools, Asm2Vec, SAFE, and Debin, that claim to capture binary semantics and show that they perform poorly (e.g., Debin F1 score dropped to 12.9% from reported 63.1%) on Cornucopia generated binaries. In summary, our exhaustive evaluation shows that Cornucopia is an effective mechanism to generate binaries for testing binary analysis techniques effectively.
暂无评论