Grammar-based transformations: attack and defence

Abstract

Purpose

This research aims to propose an attack that de-obfuscates codes by exploiting the properties of context-free grammars since it is important to understand the strength of obfuscation provided by context-free grammar-based obfuscators. In addition, the possibility of automatically generated transformations is explored.

Design/methodology/approach

As part of our empirical investigation, a development environment for obfuscating transformations is built. The tool is used to simulate a context-free obfuscator and to devise ways of reversing such transformations. Furthermore, a theoretical investigation of subset grammars and subset languages is carried out.

Findings

It is concluded that context-free grammar-based obfuscators provide limited levels of protection. Nevertheless, their application is appropriate when combined with other obfuscating techniques.

Research limitations/implications

The algorithms behave as expected on a limited number of test samples. Further work is required to increase their practicality and to establish their average reliability.

Originality/value

This research shows how a frequency analysis attack can threaten the security of code scrambled by context-free grammar-based obfuscators.

Citation

Repel, D. and Stengel, I. (2014), "Grammar-based transformations: attack and defence", Information Management & Computer Security, Vol. 22 No. 2, pp. 141-154. https://doi.org/10.1108/IMCS-09-2013-0071