Sandbox Clusters

Sandbox clusters are single node clusters running on docker containers.

Instructions#

  • Once you've created the workspace, its time to create your first cluster. Click on the Add Cluster as shown below.

Sandbox cluster!

  • Provide sandbox cluster details as per your requirement. Make sure you mount a directory your desktop on to the container running the cluster. This will enable for faster development and deployment.

Create cluster!

  • Click on the start button to initialize and run the cluster. Once the cluster has been started, you can view the resources allocated and the logs.

Cluster state!

Open Interactive Shell#

Now that we have the cluster up and running, we can interact with the cluster by opening an interacting shell from the terminal using the following command

docker exec -it <container-id> /bin/bash

In order to generate the command, click on the Open Terminal button, located at right top side of the cluster dashboard. On clicking of that button, you will be able to get the command to open interactive shell as shown below.

Docker open terminal command!

Interact with Spark and Hadoop#

Once you've open the interactive shell for the cluster, you can run spark-shell and hdfs commands to explore Spark and Hadoop respectively.

Spark shell!

It's time to automate the deployment of Spark job from the Gigahex application. Let's do it in the next section.