An analysis of system call set extraction tools on configurable Linux binaries

Abstract

System calls are a primary way in which applications to communicate with the kernel. This is to allow them to perform sensitive tasks, however, an application will typically not require all of the system calls available to function properly. Despite this, the Linux kernel allows a program to perform any system call it wishes. This is bad for security, as it allows an attacker full access to the kernel after gaining code execution in a vulnerable program. By extracting a minimal set of system calls for a given program, we can sandbox it and only allow those system calls to be executed, greatly reducing the attack surface. In this paper, we analyze existing solutions that address system call set extraction. In particular, we will focus on applying these to configurable binaries. That is, binaries which can be compiled with a variety of different settings. For this paper, we have chosen to analyze cat as a minimal example, and busybox as the configurable application. We compile busybox in the following configurations, among variations: the default configuration, a configuration containing a minimal set of features and a configuration containing a maximal set of features. We analyze the performance of the tools Binalyzer, Sysfilter and Confine on these binaries. We see that Confine has significantly worse performance than both Binalyzer and Sysfilter. We also see that Sysfilter has better performance than Binalyzer when the complexity of the busybox binary is increased. We conclude that Sysfilter outperforms Binalyzer on binaries without debug symbols, while the opposite is true when performing analysis on binaries with debug symbols.