obar1
1/23/2018 - 1:42 PM

Compile Hadoop-2.8

Compile Hadoop-2.8

Compile Hadoop-2.8 on Ubuntu 16.04

  1. Install some dependencies:

    sudo apt -y install gcc*
    sudo apt -y install cmake
    sudo apt -y install glibc-headers
    sudo apt -y install gcc-c++
    sudo apt -y install zip-devel
    sudo apt -y install openssl-devel
    sudo apt -y install svn
    sudo apt -y install git
    sudo apt -y install ncurses-devel 
    sudo apt -y install lzo-devel
    sudo apt -y install autoconf
    sudo apt -y install libtool
    sudo apt -y install automake
    
  2. Install protobuf:

    wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
    tar zxvf protobuf-2.5.0.tar.gz
    cd protobuf-2.5.0
    ./configure
    make 
    make check
    sudo make install
    sudo ldconfig
    
  3. Compile Hadoop:

    wget http://apache.spinellicreations.com/hadoop/common/hadoop-2.8.1/hadoop-2.8.1-src.tar.gz
    tar -zxvf hadoop-2.8.1-src.tar.gz
    cd hadoop-2.8.1-src
    mvn package -Dmaven.javadoc.skip=true -Pdist,native -DskipTests -Dtar
    
  4. The compiled hadoop tarball locates at hadoop-2.8.1-src/hadoop-dist/target/hadoop-2.8.1.tar.gz