OAP Developer Guide
This document contains the instructions & scripts on installing necessary dependencies and building OAP modules. You can get more detailed information from OAP each module below.
- SQL Index and Data Source Cache
- PMem Common
- PMem Spill
- PMem Shuffle
- Remote Shuffle
- OAP MLlib
- Native SQL Engine
Building OAP
Prerequisites
We provide scripts to help automatically install dependencies required, please change to root user and run:
# git clone -b <tag-version> https://github.com/oap-project/oap-tools.git
# cd oap-tools
# sh dev/install-compile-time-dependencies.sh
Note: oap-tools tag version v1.1.0-spark-3.0.0
corresponds to all OAP modules' tag version v1.1.0-spark-3.0.0
.
Then the dependencies below will be installed:
Run the following command to learn more.
# sh dev/scripts/prepare_oap_env.sh --help
Run the following command to automatically install specific dependency such as Maven.
# sh dev/scripts/prepare_oap_env.sh --prepare_maven
- Requirements for Shuffle Remote PMem Extension
If enable Shuffle Remote PMem extension with RDMA, you can refer to PMem Shuffle to configure and validate RDMA in advance.
Building
OAP is built with Apache Maven and Oracle Java 8.
To build OAP package, run command below then you can find a tarball named oap-$VERSION-bin-spark-$VERSION.tar.gz
under directory $OAP_TOOLS_HOME/dev/release-package
.
$ sh $OAP_TOOLS_HOME/dev/compile-oap.sh
Building specified OAP Module, such as sql-ds-cache
, run:
$ sh $OAP_TOOLS_HOME/dev/compile-oap.sh --sql-ds-cache