W elcome to the RDMA for Apache Hadoop (High-Performance Design of Apache Hadoop over RDMA-Enabled Interconnects) project, lead by Network-Based Computing Laboratory of The Ohio State University. The RDMA for Apache Hadoop package is a derivative of Apache Hadoop. This package can be used to exploit performance on modern clusters with RDMA-enabled interconnects for BigData applications. Major features of this package include:
- Based on Apache Hadoop 1.2.1
- Compliant with Apache Hadoop 1.2.1 APIs and applications
- High performance design with native InfiniBand and RoCE support at the verbs-level for HDFS, MapReduce, and RPC components
- Easily configurable for native InfiniBand, RoCE and the traditional sockets-based support (Ethernet and InfiniBand with IPoIB)
RDMA-Apache-Hadoop 0.9.9 (based on Apache Hadoop 1.2.1) with native InfiniBand and RoCE support for HDFS, MapReduce, and RPC, integrated with other Hadoop components and easy configuration among InfiniBand-RDMA, RoCE and sockets is available. [more]
Keynote Talk on Accelerating Data Mananagement and Processing on Clusters with RDMA-Enabled Interconnects at ADMS '14 conference. Past Keynote Talk on Big Data/RDMA for Apache Hadoop presented at Big Data Technology Conference.
Tutorial on Big Data and RDMA for Apache Hadoop to be presented at Hot Interconnect 2014 conference and IEEE Cluster 2014 Conference . Past tutorials presented at Int'l Symposium on Computer Architecture (ISCA '14) , Int'l Symposium on Cluster, Cloud and Grid Computing (CCGrid '14), Architectural Support for Programming Languages and Operating Systems (ASPLOS '14) , and High Performance Computer Architecture (HPCA '14) .