Big data computing evolved two decades ago from parallel batch processing based on the MapReduce programming model. Since then, many sophisticated big-data systems and programming models have been developed. Many of these big-data solutions are developed in Java because of its type safety and rich runtime libraries.
Project Hermes is providing two levels of abstraction with two open-source libraries: Infinileap and hadroNIO. Infinileap provides a Java interface to UCX which is a native library supporting different networks, including InfiniBand and (converged) Ethernet. While Infinileap is very efficient, it is not transparent for Java applications. The latter is achieved by hadroNIO which is offering a transparent Java NIO socket interface based on JUCX/Infinileap. The overall architecture of Hermes is shown below.
Infinileap is providing a Java interface to UCX using the Foreign Function and Memory API, see below.
HadroNIO is a Java library built on top of Infinileap providing a transparent socket interface, see below.
Filip Krakowski (Infinileap)
Fabian Ruhland (hardioNIO)
Prof. Dr. Michael Schöttner
Project Hermes is sponsored by Oracle, San Jose, USA and has received an Oracle for Research Grant from Oracle, UK