More Cluster-Aware Features In Diffusion 6.4 and 6.5

John Ziamos - May 14, 2020

When your application delivers real-time data, high availability is important. In production, you will probably want to take advantage of Diffusion™’s clustering capabilities. By clustering multiple Diffusion servers, you can add resilience to your application and keep your data flowing even through a hardware failure.

What is cluster awareness?

In earlier versions of Diffusion, not every Diffusion feature was cluster-aware. In other words, a client attached to a server could only perform operations on that individual server, rather than the whole cluster.

For example, a client can use the client control feature to close a session on the server it is connected to, but in Diffusion 6.4 and earlier, the operation can’t affect sessions on another server in the cluster.

Many Diffusion features are already cluster-aware. An example is topic replication. With topic replication enabled, a topic update is applied to all servers in the cluster, so that the topic tree is mirrored in each server. This allows a load balancer to direct a client to any server in the cluster and deliver the same stream of topic updates.

Expanded cluster awareness in Diffusion 6.4

In newer versions of Diffusion, we’ve expanded the number of features which are cluster-aware.

In Diffusion 6.4, we added cluster awareness to control authentication handlers (used to authenticate client connections or restricted actions) and missing topic notifications.

What does cluster awareness mean for these features? If an authentication handler or missing topic notification handler is not registered on the server that receives the request, it will be forwarded to another server in the same cluster (if it has one registered). So if you have a three-server cluster, instead of needing three handlers each registered on a different server, you only need one for the whole cluster.

 

6.5 brings cluster awareness to even more features

In 6.5 we will also be making request-response messaging and many client control operations cluster-aware.

Sending a request to a request-response handler benefits from cluster awareness in the same way as authentication and missing topic notification.

For client control operations, if a client specifies a session ID, the target session need not be connected to the same server. The command will be routed to the server in the cluster that hosts that specified session. Also, when specifying a session filter, the command will be applied to all matching sessions across the cluster, rather than only to those which the sending session is connected.

The client control operations that now work across a cluster are:

  • changeRoles
  • close
  • setConflated
  • setSessionProperties
  • getSessionProperties

If a session identified by a session ID is not found on the server that the calling session is connected to, it will be routed to a server in the cluster that the target session is connected to.

In addition, version 6.5 has added the ability to use close and setConflated with a session filter instead of a session ID, enabling batch operations. A command that identifies a set of sessions using a session filter will be executed on all servers.

When to use cluster awareness

Note that the performance of local operations will always be optimal because they do not require inter-server communication. However, taking advantage of cluster awareness provides three distinct advantages:

1. Handlers which are called infrequently can be available on only a subset of your servers.
2. In the event that a local handler or client is unavailable, cluster awareness
provides further resilience.
3. Servers can be dynamically added to horizontally scale a cluster without needing to create and connect extra clients.

In summary, increased cluster awareness gives you more options to architect your Diffusion-powered application, while making it more reliable and easier to manage.


The Diffusion Intelligent Data Mesh manages, optimizes, and integrates data among devices, systems, and applications. Push Technology pioneered and is the sole provider of real-time delta data streaming™ technology that powers mission-critical business applications worldwide. Leading brands use Push Technology to fuel revenue growth, customer engagement, and business operations. The products, Diffusion® and Diffusion Cloud™, are available on-premise, in-the-cloud, or in a hybrid configuration, to fit the specific business and infrastructure requirements of the applications operating in today’s mobile obsessed, everything connected world. Learn how Push Technology can reduce infrastructure costs, and increase speed, efficiency, and reliability, of your web, mobile, and IoT application.

Learn More: View Blog Posts and Developer Resources

Share:

Check out these other resources

Info

High availability

How to replicate session, topic and configuration information between clustered Diffusion servers to increase availability and reliability

Info

Get Diffusion

Download the latest version of on-premise Diffusion

Case Study

Unlock growth & scale

Case studies showing how companies use Diffusion to deliver more data with less infrastructure