- Spark v2.0 or later
- Java 7 or later
First, you download a compiled Spark package from the Spark official web page and invoke spark-shell with a compiled Hivemall binary.
$ ./bin/spark-shell --jars hivemall-spark-xxx-with-dependencies.jar
If you would like to try Hivemall functions on the latest release of Spark, you just say
bin/spark-shellin a Hivemall package. This command automatically downloads the latest Spark version, compiles Hivemall for the version, and invokes spark-shell with the compiled Hivemall binary.
Then, you load scripts for Hivemall functions.
scala> :load resources/ddl/define-all.spark scala> :load resources/ddl/import-packages.spark