Typesafe Activator

Akka Cluster Sharding with Scala!

Akka Cluster Sharding with Scala!

typesafehub
Source
July 2, 2014
akka cluster scala sample

Illustrates sharding of event sourced actors in a cluster of Akka nodes.

How to get "Akka Cluster Sharding with Scala!" on your computer

There are several ways to get this template.

Option 1: Choose akka-cluster-sharding-scala in the Typesafe Activator UI.

Already have Typesafe Activator (get it here)? Launch the UI then search for akka-cluster-sharding-scala in the list of templates.

Option 2: Download the akka-cluster-sharding-scala project as a zip archive

If you haven't installed Activator, you can get the code by downloading the template bundle for akka-cluster-sharding-scala.

  1. Download the Template Bundle for "Akka Cluster Sharding with Scala!"
  2. Extract the downloaded zip file to your system
  3. The bundle includes a small bootstrap script that can start Activator. To start Typesafe Activator's UI:

    In your File Explorer, navigate into the directory that the template was extracted to, right-click on the file named "activator.bat", then select "Open", and if prompted with a warning, click to continue:

    Or from a command line:

     C:\Users\typesafe\akka-cluster-sharding-scala> activator ui 
    This will start Typesafe Activator and open this template in your browser.

Option 3: Create a akka-cluster-sharding-scala project from the command line

If you have Typesafe Activator, use its command line mode to create a new project from this template. Type activator new PROJECTNAME akka-cluster-sharding-scala on the command line.

Option 4: View the template source

The creator of this template maintains it at https://github.com/typesafehub/activator-akka-cluster-sharding-scala#master.

Option 5: Preview the tutorial below

We've included the text of this template's tutorial below, but it may work better if you view it inside Activator on your computer. Activator tutorials are often designed to be interactive.

Preview the tutorial

Akka cluster Sharding is useful when you need to distribute actors across several nodes in the cluster and want to be able to interact with them using their logical identifier, but without having to care about their physical location in the cluster, which might also change over time.

It can be actors representing Aggregate Roots in Domain-Driven Design terminology. These actors typically have persistent (durable) state, which can be implemented with Akka Persistence.

Cluster sharding is typically used when you have many stateful actors that together consume more resources (e.g. memory) than fit on one machine. If you only have a few stateful actors it might be easier to run them on a Cluster Singleton node.

The Blog Post Actor

To illustrate the cluster sharding feature we will use a small blog post application. To make it easy to understand it has limited capabilities, and that also makes it possible for you to get your hands dirty by adding more functionality as suggested in the end of the tutorial.

Open Post.scala.

Post is an actor representing an individual blog post, i.e. we will create a new Post actor instance for each new blog post. The actor is using event sourcing to store the changes that build up its current state.

Look at how the different messages are handled and note the similarities. An event sourced actor typically follows these steps when receiving a message:

  1. validate the incoming message
  2. create a domain event that represents the state change, and store it with persist
  3. update the actor's state inside the persist block, which is invoked after successful storage
  4. do external side effects inside the persist block

Note that if the JVM crashes right after successful storage the side effect in the last step is never executed even though the event has been stored.

For some use cases you may want to do external side effects before persisting the domain events, with the risk that the side effect was performed even though storage of the event fails.

It is recommended to encapsulate the state in an immutable class as illustrated in the Post.State class. It knows how to create a new State instance when applying the changes represented by domain events. It is important that the state updates are free from side effect, because they are applied when the actor is recovered from the persisted events. See receiveRecover.

Sharding the Blog Post Actor

Open BlogApp.scala.

To make the Post actors sharded in the cluster we need to register it to the ClusterSharding extension. See ClusterSharding(system).start in the BlogApp. Descriptions of the parameters can be found in the API documentation of ClusterSharding. This must be done on all nodes in the cluster.

The sharding is based on a hash function of the postId. That function is defined in the companion object of the Post. The postId is a String representation of a UUID.

Open Bot.scala.

The Bot actor simulates creation of blog posts by several authors.

To send messages to the identifier of the Post actor you need to send them via the shardRegion actor, which can be retrieved via the ClusterSharding extension. The sharding feature knows how to route the message and it will on demand allocate the Post actors to cluster nodes and create the actor instances. Exactly how this works under the hood is described in the documentation.

Listing of Blog Posts

Open AuthorListing.scala.

The AuthorListing actor collects a list of all published post by a specific author, i.e. one AuthorListing instance for each author.

The Persistent PostSummary messages are sent by the Post actor when a post is published.

AuthorListing is also sharded. It is registered in the BlogApp and its region is used in the Bot. It is using a hash function of the author. That function is defined in the companion object of the AuthorListing.

Test

A multi-jvm test for the blog application can be found in BlogSpec.scala. You can run it from the Test tab.

Run the Simulation

Go to the Run tab to see the running BlogApp with the Bot. In the log output you should be able to see that new blog posts are created.

BlogApp starts three actor systems (cluster members) in the same JVM process. It can be more interesting to run them in separate processes. Stop the application in the Run tab and then open three terminal windows.

In the first terminal window, start the first seed node with the following command:


<path to activator dir>/activator "runMain sample.blog.BlogApp 2551"

2551 corresponds to the port of the first seed-nodes element in the configuration. In the log output you see that the cluster node has been started and changed status to 'Up'.

In the second terminal window, start the second seed node with the following command:


<path to activator dir>/activator "runMain sample.blog.BlogApp 2552"		

2552 corresponds to the port of the second seed-nodes element in the configuration. In the log output you see that the cluster node has been started and joins the other seed node and becomes a member of the cluster. Its status changed to 'Up'.

Switch over to the first terminal window and see in the log output that the member joined. So far, the Bot has not been started, i.e. no blog posts are created.

In the third terminal window, start one more node with the following command:


<path to activator dir>/activator "runMain sample.blog.BlogApp 0"

Now you don't need to specify the port number, 0 means that it will use a random available port. It joins one of the configured seed nodes. Look at the log output in the different terminal windows.

The Bot is started on nodes with port not equal to 2551 or 2552, i.e. now it has not been started. You should be able to see log output of the Bot that generates blog posts.

Take a look at the logging that is done in Post, AuthorListing and Bot. Identify the corresponding log entries in the 3 terminal windows.

Shutdown the node with port 2552 (second terminal window) with ctrl-c. Observe how the other nodes detect the failure and remove the node from the cluster. Also note that requests to the AuthorListing for a specific author that previously was located on the node with port 2552 will be failed over to one of the other nodes. Look for the log message starting with "Post added to".

You can also start even more nodes with the command:


<path to activator dir>/activator "runMain sample.blog.BlogApp 0"

For each additional node another Bot is also started.

Note that this sample runs the shared LevelDB journal on the node with port 2551. This is a single point of failure, and should not be used in production. A real system would use a distributed journal.

The files of the shared journal are saved in the target directory and when you restart the application the state is recovered. You can clean the state with:


<path to activator dir>/activator clean

Hands-on Exercises

Here is a list of improvements to the blog application that you can try yourself:

  • Enhance the functionality of the event sourced Post actor. Edit title. Change author. Adding and removing comments.
  • Note that the Bot naively assumes that each step of creating, changing and publishing a blog post is succesful. If some step fails it will get stuck in the "wrong state". Add acknowlegement messages for the operations in the Post actor and handle the failure scenarios in the Bot.
  • Create a CQRS query side by using a persistent view of the AuthorListing processor.
  • Replace the shared LevelDB journal with a real replicated journal. Pick one from the community plugins.
comments powered by Disqus