Integrating Data & Systems

  • Ingest Data From Devices & Applications to MQTT

    Concerns:

    • Back-end pipeline using MQTT for messaging
    • Large numbers of web/mobile applications producing data for ingest
    • Handle volatile connectivity of apps (mobile networks)

    How Diffusion helps:

    • Binary deltas – drastically reduce bandwidth needed for data broadcast = lower data costs
    • 1st party Diffusion SDKs for IoT – JS, iOS, Android
    • Diffusion as middleware – enables large numbers of connections from devices without needing large investment in back-end infrastructure
    • Reliable reconnection – no messages lost when devices become disconnected
  • Extend Data From MQTT to Web/Mobile Applications

    Concerns:

    • Back-end pipeline using MQTT for messaging
    • MQTT to Web/Mobile, if supported at all, is primarily polling-based, not designed for large numbers of connections
    • Handle volatile connectivity of apps (mobile networks)

    How Diffusion helps:

    • MQTT Adapter – easily extends MQTT topics to web/mobile
    • 1st party mobile & web SDKs
    • Pub/Sub & Binary deltas – less bandwidth needed for sending messages reduces data cost for end-users
    • Diffusion as middleware – designed for high numbers of connections (hundreds of thousands), reduces load on back-end systems
    • Automatic reconnection – receive messages that were broadcast while app is disconnected
  • Ingest Data From Devices & Applications to JMS

    Concerns:

    • Back-end pipeline using JMS for messaging
    • Large numbers of web/mobile applications producing data for ingest
    • Handle volatile connectivity of apps (mobile networks)

    How Diffusion helps:

    • Binary deltas – drastically reduce bandwidth needed for data broadcast = lower data costs
    • 1st party Diffusion SDKs for IoT – JS, iOS, Android
    • Diffusion as middleware – enables large numbers of connections from devices without needing large investment in back-end infrastructure
    • Reliable reconnection – no messages lost when devices become disconnected
  • Extend Data From JMS to Web/Mobile Applications

    Concerns:

    • Back-end pipeline using JMS for messaging
    • JMS to Web/Mobile, if supported at all, is primarily polling-based, not designed for large numbers of connections
    • Handle volatile connectivity of apps (mobile networks)

    How Diffusion helps:

    • JMS Adapter – easily extends JMS topics to web/mobile
    • 1st party mobile & web SDKs
    • Pub/Sub & Binary deltas – less bandwidth needed for sending messages reduces data cost for end-users
    • Diffusion as middleware – designed for high numbers of connections (hundreds of thousands), reduces load on back-end systems
    • Automatic reconnection – receive messages that were broadcast while app is disconnected
  • Ingest Data From Devices & Applications to Kafka

    Concerns:

    • Back-end pipeline using Kafka for data processing / distribution
    • Large numbers of web/mobile applications producing data for ingest to Kafka
    • Handle volatile connectivity of apps (mobile networks)

    How Diffusion helps:

    • Binary deltas – drastically reduce bandwidth needed for data broadcast = lower data costs
    • 1st party Diffusion SDKs for Mobile/Web – JS, iOS, Android
    • Diffusion as middleware – enables large numbers of connections from devices without needing large investment in back-end infrastructure
    • Reliable reconnection – no messages lost when devices become disconnected
  • Extend Data From Kafka to Web/Mobile Applications

    Concerns:

    • Back-end pipeline using Kafka for data processing / distribution
    • Data in Kafka needs to be sent to large numbers of web/mobile applications
    • Kafka has no official JS/Mobile client – 3rd party solutions are polling-based, not designed for large numbers of connections
    • Data consistency & message ordering needs to be preserved
    • Handle volatile connectivity of apps (mobile networks)

    How Diffusion helps:

    • Kafka Adapter – easily extends Kafka topics to web/mobile
    • Pub/Sub & Binary deltas – less bandwidth needed for sending messages reduces data cost for end-users
    • Diffusion is designed for high numbers of connections (hundreds of thousands), reduces load on back-end systems
    • Topic ordering – all updates for a topic are sent to subscribers in-order
    • Automatic reconnection – receive messages that were broadcast while app is disconnected
  • Transform REST Endpoints to Streaming

    Concerns:

    • Existing back-end systems provide data via RESTful APIs
    • Continuous polling for new data adds unnecessary bandwidth costs and consumes device battery
    • Changes in data models should be received immediately
    • Scaling consumers introduces high-levels of load on back-end infrastructure

    How Diffusion helps:

    • REST adaptor automatically creates streaming topics from RESTful endpoints
    • Single polling client (REST adaptor) reduces load on back-end systems with Diffusion handling end-user device connections
    • Broad variety of 1st party Diffusion SDKs for mobile platforms (Java, JS, iOS, Android)
    • Pub/Sub model allows subscription to real-time changes in data across multiple topics
  • Extend Real-Time Streaming to MuleSoft

    Concerns:

    • Enterprise integrations managed with MuleSoft
    • Back-end data sources need to be extended to web/mobile platforms
    • No out-of-the-box support for real-time streaming

    How Diffusion helps:

    • MuleSoft connector – use MuleSoft to broadcast data using Diffusion
    • 1st party Diffusion SDKs for web and mobile (JS, iOS, Android) with WebSocket streaming
    • Data agnostic – Native Client SDK support for JSON & Binary data
  • Integrate Disparate Data Sources

    Concerns:

    • Data is produced by multiple systems in various formats
    • Want to avoid tightly coupling producers and consumers of data
    • Requires a multi-stage data pipeline architecture

    How Diffusion helps:

    • Pub/Sub model – decouples producers from consumers of data.
    • Native support for Binary & JSON values
    • Broad variety of 1st party Diffusion SDKs for multiple platforms (Java, .Net, C, JS, iOS, Android)
    • Dynamic topic creation allows clients to receive, transform, and then publish data feeds for easy management of multi-stage data transformations
  • Extend Legacy Data Sources

    Concerns:

    • Data stored in legacy systems needs to be delivered to modern (web/mobile) applications
    • Data is in proprietary or outdated formats
    • Want to avoid tightly coupling consuming application with the data source

    How Diffusion helps:

    • Pub/Sub – decouples producers from consumers of data. Build modern consumer applications on Diffusion, and then iteratively replace data-sources over time without affecting front-end.
    • 1st class support for Binary data – no need to worry about parsing/transformation at producer side.
    • C Client SDK enables low-level bindings to be easily written for legacy systems
    • 1st party Client SDKs for web & mobile platforms for easy consumption of data