Low-overhead Online Code Transformations

Since their introduction in the mid-twentieth century, compilers have evolved from complex ad-hoc algorithms to well-understood constructs with clearly defined design goals and stages. These stages were established many years ago, and have remained relatively static since, trapping the world into viewing the compilation of programs written in high-performance languages like C, C++ and Fortran as something done prior to, and fundamentally distinct from, running the program. This status quo has held because the ability to perform online code transformations — changing executing native programs — has always carried significant performance overhead.

This dissertation proposes and investigates the implications of a new approach to enabling online code transformations, wherein the dynamic compiler runs asynchronously and in parallel to the running program. This approach to dynamic compilation keeps the runtime overhead of possessing an online code transformation capability to near-zero, allowing programs to execute with this capability at near-native speed. Having the capability to continuously perform online code transformations opens up a new design space for native programs, as such programs no longer need to have a fixed implementation throughout their execution.

Many opportunities become available when online code transformations are always available. This dissertation describes two novel applications of the proposed approach to online code transformations, resulting in best-in-class solutions to two pressing problems facing modern computer scientists. First, we leverage online code transformations to significantly increase the utilization of multicore datacenter servers, reducing the number of servers needed to host latency-critical web services and by extension mitigating the cost and environmental impacts of operating datacenters. Our technique works by strategically injecting software caching hints into a running application, allowing the applications housed on the server to cooperatively use shared server resources far more effectively than prior techniques that lack the online code transformation capability. Second, we employ online code transformation to build an operating system that is resistant to code reuse attacks, a widespread technique used by malicious attackers in subverting and hijacking the execution of modern software. Our technique works by building the proposed online code transformation capability into the operating system (OS). Our design allows the OS to efficiently, transparently and continuously re-randomize code inside running applications, invalidating attackers' assumptions as to the location of code in memory that are needed to successfully execute code reuse attacks.