March 24, 2021 Sanjai Marimadaiah

Full Diffusion 6.6 Release Is Here: Introducing The Intelligent Event Data Platform

Push Technology is proud to share the enhanced product vision through the release of Diffusion 6.6.

We are introducing the new name Diffusion Intelligent Event Data Platform. We classify the capabilities of Diffusion into 3 key components: Data Gateway, Data Wrangling and Data Distribution.

This is the full Diffusion 6.6 release. Some features added since 6.5 were previously available in the 6.6 Preview 1 and Preview 2 releases; in some cases these features have been substantially altered and improved since the preview.

Data Gateway

Data Gateway component facilitates the consumption of data from a variety of sources and in disparate formats and publishing the processed event data into Diffusion Platform. Data Gateway includes adapters to connect with a variety of data sources. The vision for Data Gateway includes capabilities for transformation of formats, computation to add business logic and more.

Following are the capabilities added to Data Gateway component in Diffusion 6.6:

Improved Kafka adapter

You can now control these adapters and visualize their activity using the management console.

The Kafka adapter in this release offers more configuration options than the preview version, including the ability to configure how imported topics are structured, and to import topics that match a regular expression. The adapter can translate data from Kafka topics to Diffusion topics, and from Diffusion to Kafka.

For more information, see Kafka adapter docs, our blog post about the new adapter, and our tutorial about integrating Kafka and Diffusion.

Improved JMS adapter

The Java Message Service (JMS) adapter has been updated to use a JSON configuration format.

Support for Diffusion topic subscription is added, which allows the updates to Diffusion topics to be sent to JMS destinations.
The adapter can be visualized and monitored in the Diffusion management console as well.

See JMS adapter docs for details.

MQTT support

Diffusion now supports the popular MQTT messaging protocol, enabling MQTT devices to publish and subscribe to Diffusion topics.

Now you can gather and distribute real-time data to Internet of Things (IoT) devices and other remote hardware, while still making use of Diffusion’s advanced data-wrangling and security capabilities. There is no need to install any Diffusion code on the remote devices.

MQTT is supported for production use. See MQTT support docs and our MQTT tutorial for full details.

Data Wrangling

Data Wrangling component provides the capabilities to design the Topic Tree structure and access controls to route the event data for further distribution. These activities can be performed via the Management Console or the SDKs. The Data Wrangling component has rich set of capabilities to dynamically transform the event data as well as the Topic Tree design.

Following are the capabilities added to Data Wrangling component in Diffusion 6.6:

Topic view preserve topics option

Topic views are a dynamic way to map and transform a set of topics, creating reference topics in another part of the topic tree.
Some topic views can create reference topics with paths that depend on the values of source topics. This means that if the value used to determine a reference topic path changes, the reference topic gets removed.

There is now a new option to preserve reference topics derived from a value, even when the value changes. This makes the set of reference topics generated more stable.

See the preserve topics clause docs for details.

Topic view separator option

By default, when a topic view derives a reference topic path from a source topic value, if the value contains a / character, it is treated as a path separator.

For example, a value like “Gold/Silver Index” could create a branch called “Silver Index” under a branch called “Gold”, making the topic path longer than required.

Diffusion 6.6 introduces a new option to replace / with a string of your choice when creating reference topics.

See the separator clause docs for details.

Topic view insert

Topic view inserts are a new addition to the data processing capabilities of topic views. Topic views enable you to mirror selected source topics to another part of the topic tree, creating reference topics.

With a topic view insert, you can now merge data from topics other than the selected source topic into JSON reference topics. You can insert whole values, or partial data specified with JSON pointers.

Advanced features include the ability to derive paths from values, and chaining multiple inserts.

For more information, see the insert clause docs and our blog post with more detailed use cases.

Time series enhancements

You can now update a time series topic via the standard topic update API, treating a time series topic as if it were a single topic with the same event type as the time series. This means that when you update time series topics, you can now use features like update constraints, update streams and the addAndSet operation.

In addition, you can now create a time series event with a custom timestamp of your choice, instead of one based on the current time. You could use this to load historical data into a time series topic, or for testing purposes.

Data Distribution

Data Distribution component facilitates the massive scale of event data transmission around the globe in milliseconds, with secure access control to assure reliable delivery of the exact data, in the correct format that each recipient requires.

Following are the capabilities added to Data Distribution component in Diffusion 6.6:

JavaScript SDK client: dynamic library loading

To minimize application size and loading times, the JavaScript client is now available as a collection of modular bundles.
Your application can load a smaller core client bundle, and load additional features dynamically as required.

See JavaScript SDK docs for details.

Python SDK

The Python SDK enables you to develop Diffusion clients in Python, with support for core features including subscribing and publishing. Some features available in other SDKs are not yet supported in Python.

See the Python SDK overview for details.

Other Improvements

  • Session filters now support a new IN operator, so you can filter sessions based on a list of properties.
  • The Android SDK has been updated to take advantage of the latest desugaring support in Java 8+.
  • Update streams now respect topic conflation settings and can be used to update topics that with DONT_RETAIN_VALUE set to true.
  • The server will use less CPU when update traffic is very low.

Diffusion Management Console

Diffusion Management console brings all the 3 components together to deliver the single view of the Diffusion Intelligent Event Data Platform.

Following are the capabilities added to the management console component in Diffusion 6.6:

Single sign-on to the management console

The Diffusion management console now supports integration with third-party single sign-on systems.
See Configuring the Diffusion management console for details.

Cluster visualization

When you’re using Diffusion’s high-availability capabilities, you can now see the status of your server clusters from within the management console.


Whether you are a new customer or are already experiencing the power of Diffusion, we hope you enjoy the rich functionality of Diffusion 6.6 as part of your event-driven application. It is available for immediate download today.

 

diffusion

Enjoy the rich functionality of Diffusion 6.7 as part of your event-driven application.

diffusion
Quick Start Guide

Step-by-Step guides to get started

diffusion
Diffusion Cloud

SaaS offering to focus on business

diffusion
Diffusion On-Premise

A holistic pub-sub platform for real-time apps