GAON: General-purpose Application Offload to Near-Network Processors

Emerging intelligent bump-in-the-wire network adapters, iNICs, are increasingly
deployed for accelerating custom network functions in large-scale data center and
cloud environments such as Microsoft Azure. However, their potential to speed up
network-intensive server applications remains largely unexplored due to the lack
of appropriate programming models and OS abstractions.

GAON is a framework that facilitates accelerating network applications using
iNICs. GAON introduces an ikernel abstraction that encapsulates the code and
state of an iNIC-resident application task. The application dynamically invokes
the ikernel by attaching it to a network socket, thereby rerouting the socket
traffic to the ikernel. GAON can securely execute multiple ikernels of different
applications, isolating ikernel state and network traffic, arbitrating access to
network from ikernels and CPU applications, and providing protected access to the
ikernel state from the owning CPU application.

We prototype GAON on Mellanox Innova 40 Gbps iNICs and develop several
accelerated versions of realistic server workloads. For example, the transparent
key-value store cache
ikernel serves 31Mtps (70% of the line rate) at 5 µs 99th percentile latency,
seamlessly improving the host performance by up to 55— and 2.3— for 98%
and 60% cache hit rate respectively, with almost no modification to the original
memcached server running on the same machine.
Mark Silberstein is an Assistant Professor at the department of Electrical
Engineering, Technion – Israel Institute of Technology. Mark is interested in
accelerator-rich systems and OS abstractions and services for and on
accelerators, with the ultimate goal to break out of the CPU-centric OS models
(HotOS17). His works include GPUfs (ASPLOS13), GPUnet (OSDI14) and GPUrdma
libraries for native file system, networking and RDMA support for GPU programs
and application-managed virtual memory for GPUs (ISCA16) and SGX enclaves
(EuroSYS17).