Architecture-independent matching of stripped binary code files using BERT and a Siamese neural network

Abstract

The proliferation of IoT devices brings many cyber security challenges. Identifying executable code with known vulnerabilities is one of them, this despite the fact that open source code is commonly used in IoT firmware. Factors that contribute to this challenge include the high usage of heterogeneous architectures, as well as non-standard toolsets and compilers when developing IoT firmware. To address this issue, this work examines the latest research in bi-nary code matching. It concludes that the research does not adequately address the current cyber security issues incurred by IoT devices and proposes a new method of binary code matching based on techniques and methods commonly seen in Natural Language Processing (NLP). An artefact using Google’s BERT and a custom bi-directional LSTM Siamese network is developed and tested to demonstrate the viability of this new method. The BERT model was pre-trained using the code sections of binary executables compiled for the ARM architecture. It achieved scores of 89.1% and 98.0% in the key metrics of masked_lm_accuracy and next_sentence_accuracy respectively. This pre-trained BERT model was used to extract embeddings from the binary files’ code sections in order to train and validate the Siamese network. The Siamese network achieved an average rate of approximately 80% on the task of match-ing the stripped code sections of binary files compiled by two separate open source projects. This compares favorably to the 0% accuracy achieved by the fuzzy hashing algorithms SSDEEP and SDHASH.