Spark

安装

  • 安装JAVA

    $ java -version
    
  • 安装Scala

    $ wget https://downloads.lightbend.com/scala/2.12.6/scala-2.12.6.rpm
    $ sudo rpm -ivh scala-2.12.6.rpm
    $ scala -version
    
  • 安装Spark

    $ wget http://mirrors.hust.edu.cn/apache/spark/spark-2.3.1/spark-2.3.1-bin-hadoop2.7.tgz
    $ tar xvf spark-2.3.1-bin-hadoop2.7.tgz
    $ sudo mv spark-2.3.1-bin-hadoop2.7/* /usr/local/spark
    
    $ vim ~/.bashrc
    export PATH = $PATH:/usr/local/spark/bin
    $ source ~/.bashrc
    
    $ spark-shell