On the infeasibility of modeling polymorphic shellcode Re-thinking the role of learning in intrusion detection systems

作     者：Song, Yingbo Locasto, Michael E. Stavrou, Angelos Keromytis, Angelos D. Stolfo, Salvatore J. 

作者机构：Columbia Univ Dept Comp Sci New York NY 10027 USA George Mason Univ Dept Comp Sci Fairfax VA 22030 USA 

出 版 物：《MACHINE LEARNING》 (机器学习)

年 卷 期：2010年第81卷第2期

页      面：179-205页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Air Force Research Laboratory [FA8750-06-2-0221] Army Research Office [W911NF0610151] NSF [06-27473] 

主　　题：Shellcode Polymorphism Metrics Blending 

摘      要：Current trends demonstrate an increasing use of polymorphism by attackers to disguise their exploits. The ability for malicious code to be easily, and automatically, transformed into semantically equivalent variants frustrates attempts to construct simple, easily verifiable representations for use in security sensors. In this paper, we present a quantitative analysis of the strengths and limitations of shellcode polymorphism, and describe the impact that these techniques have in the context of learning-based IDS systems. Our examination focuses on dual problems: shellcode encryption-based evasion methods and targeted blending attacks. Both techniques are currently being used in the wild, allowing real exploits to evade IDS sensors. This paper provides metrics to measure the effectiveness of modern polymorphic engines and provide insights into their designs. We describe methods to evade statistics-based IDS sensors and present suggestions on how to defend against them. Our experimental results illustrate that the challenge of modeling self-modifying shellcode by signature-based methods, and certain classes of statistical models, is likely an intractable problem.