Typesafe Activator

Spark Streaming with Scala

Spark Streaming with Scala

jaceklaskowski
Source
June 5, 2014
spark streaming akka scala scaladays2014

Apache Spark (http://spark.apache.org/) is a fast and general engine for large-scale data processing. This Typesafe Activator template demonstrates Apache Spark (http://spark.apache.org) for data streaming with Scala and Akka using the Spark Streaming (http://spark.apache.org/docs/latest/streaming-programming-guide.html) extension.

How to get "Spark Streaming with Scala" on your computer

There are several ways to get this template.

Option 1: Choose spark-streaming-scala in the Typesafe Activator UI.

Already have Typesafe Activator (get it here)? Launch the UI then search for spark-streaming-scala in the list of templates.

Option 2: Download the spark-streaming-scala project as a zip archive

If you haven't installed Activator, you can get the code by downloading the template bundle for spark-streaming-scala.

  1. Download the Template Bundle for "Spark Streaming with Scala"
  2. Extract the downloaded zip file to your system
  3. The bundle includes a small bootstrap script that can start Activator. To start Typesafe Activator's UI:

    In your File Explorer, navigate into the directory that the template was extracted to, right-click on the file named "activator.bat", then select "Open", and if prompted with a warning, click to continue:

    Or from a command line:

     C:\Users\typesafe\spark-streaming-scala> activator ui 
    This will start Typesafe Activator and open this template in your browser.

Option 3: Create a spark-streaming-scala project from the command line

If you have Typesafe Activator, use its command line mode to create a new project from this template. Type activator new PROJECTNAME spark-streaming-scala on the command line.

Option 4: View the template source

The creator of this template maintains it at https://github.com/jaceklaskowski/spark-activator#master.

Option 5: Preview the tutorial below

We've included the text of this template's tutorial below, but it may work better if you view it inside Activator on your computer. Activator tutorials are often designed to be interactive.

Preview the tutorial

A starter application with Apache Spark

Apache Spark is a fast and general engine for large-scale data processing.

This tutorial illustrates how one can use Apache Spark to...FIXME.

The Mission

The mission is to use...FIXME.

Define ...

A job is...(you can find the code in the HelloSpark file).


// Scala code

case object X
case class XX(y: String)
case class XXX(y: String)
        
You use ....(...) to show code snippets and italics to have a test italiciazed.

Run the App

Congratulations!!!

Now you have almost completed the tutorial and written a simple Apache Spark application. If you have not looked at the full sample then now is a good time to do so.

Let's give the sample application a shot.

In Run, select the application to run from the drop-down list, and select Start. Feel free to modify, compile and re-run the sample.

Run the App

Congratulations!!!

Now you have almost completed the tutorial and written a simple Akka application. If you have not looked at the full sample then now is a good time to do so.

Let's have some fun and run it.

In Run, select the application to run from the drop-down list, and select Start. Feel free to modify, compile and re-run the sample.

Conclusion

This tutorial has introduced how Apache Spark can be used for...FIXME.

Next Steps

The Spark Documentation offers Setup instructions, programming guides, and other documentation.

If you have questions don't hesitate to post them to the user@spark.apache.org mailing list.
or contact the author of the activator.

comments powered by Disqus