I will cover Spark Standalone and Spark on YARN installation examples in this hour because these are the most common deployment modes in use today. In the case of Spark on YARN, this typically involves deploying Spark to an existing Hadoop cluster. In each case, you would need to establish a working YARN or Mesos cluster prior to installing and configuring Spark. Spark on YARN and Spark on Mesos are deployment modes that use the resource schedulers YARN and Mesos respectively. Spark Standalone includes everything you need to get started. With Spark Standalone, you can get up an running quickly with few dependencies or environmental considerations. The term “standalone” simply means it does not need an external scheduler.
The term can be confusing because you can have a single machine or a multinode fully distributed cluster both running in Spark Standalone mode. Spark Standalone refers to the built-in or “standalone” scheduler. There are three primary deployment modes for Spark:
#Install apache spark centos 6 how to
I will also cover how to deploy Spark on Hadoop using the Hadoop scheduler, YARN, discussed in Hour 2.īy the end of this hour, you’ll be up and running with an installation of Spark that you will use in subsequent hours. This hour covers the basics about how Spark is deployed and how to install Spark.
Now that you’ve gotten through the heavy stuff in the last two hours, you can dive headfirst into Spark and get your hands dirty, so to speak. What the different Spark deployment modes are
0 kommentar(er)
