Overview

W elcome to the RDMA for Apache Hadoop (High-Performance Design of Apache Hadoop over RDMA-Enabled Interconnects) project, lead by Network-Based Computing Laboratory of The Ohio State University. The RDMA for Apache Hadoop package is a derivative of Apache Hadoop. This package can be used to exploit performance on modern clusters with RDMA-enabled interconnects for BigData applications. Major features of this package include:

  • Based on Apache Hadoop 1.2.1
  • Compliant with Apache Hadoop 1.2.1 APIs and applications
  • High performance design with native InfiniBand and RoCE support at the verbs-level for HDFS, MapReduce, and RPC components
  • Easily configurable for native InfiniBand, RoCE and the traditional sockets-based support (Ethernet and InfiniBand with IPoIB)

Announcements


RDMA-Apache-Hadoop 0.9.9 (based on Apache Hadoop 1.2.1) with native InfiniBand and RoCE support for HDFS, MapReduce, and RPC, integrated with other Hadoop components and easy configuration among InfiniBand-RDMA, RoCE and sockets is available. [more]

Keynote Talk on Accelerating Data Mananagement and Processing on Clusters with RDMA-Enabled Interconnects at ADMS '14 conference. Past Keynote Talk on Big Data/RDMA for Apache Hadoop presented at Big Data Technology Conference.