When I joined Push I was set a challenge – how could I more tangibly demonstrate Diffusion’s core capabilities and performance? I was game…
An essential requirement for engineers at Push Technology is, of course, solid knowledge of the company’s products. At core, this means the configuration of and integration with our real time data distribution software, Diffusion™. The best way to attain this knowledge is by implementing a solution based around Diffusion and, hence, the rite of passage that is demo creation.
Having been asked to specify, design and implement a demo my first challenge was to find a source of data that could trigger a suitably demonstrable real time output.
Real Real Time
I had already seen the kinds of data source that had been used by others. These included stock market feeds, flight positions and meteorological measurements. They all suffered from one or more of the same flaws in respect of real time demonstrability:
Generated by a third party and often subject to specific times when they’re active. We want something real time (available at any time of the day or week).
Not changing very fast. We want something real time (subject to constant change).
Lack of Immediacy
No way to see cause and effect. We want something real time (where latency is clear to see).
In the past I have done quite a bit of work interfacing external control systems with video displays. Inevitably this has meant integrating with commodity desktop hardware that’s used more typically for gaming (e.g. joysticks). It is possible to poll this hardware at reasonably high frequencies (typically anywhere from 60Hz up to 150Hz or beyond) with minimal latency.
Perfect! Gaming hardware can be controlled by us (always available), brings an element of excitement (always changing) and demonstrates immediacy (absolute feedback).
Real World: The Formula One Context
In respect of creating a demonstration, especially of a product which has so many potential uses as Diffusion, it’s also beneficial to identify a real world application for which the demo might provide a foundation.
In the back of my mind, while considering gaming devices as a source of real time data, was the world of Formula One. The F1 engineering team in the pits receive a constant stream of data from the car. Wouldn’t it be cool for race fans to be able to see some of that same data in real time as the race progresses?
For a fan sitting beside the track this could provide a fascinating additional layer to their experience. For those watching on their television at home there are additional challenges due to the latency introduced by the many layers of digital video transmission technology involved in getting ‘live’ television in to the living room. Essentially, there would be a requirement to buffer and delay the data stream being received from Diffusion to match the TV pictures.
Input: The Publisher
A Thrustmaster Ferrari Challenge Wheel steering device was selected for development purposes. Not only are these cheap and readily available, but they’re also well supported with native drivers for Microsoft Windows.
DirectInput is one of the APIs offered by Microsoft as part of DirectX. Natively, DirectX is integrated with by application developers using C++. While this is a popular language for many, it’s not the most accessible for rapid prototyping or for producing clear and legible code for demo and example purposes. For these reasons, plus the fact that I’m not a particular fan of developing in C++, I chose to write my publisher in C# targeting the .NET framework. This made a lot of sense in my chosen Microsoft Windows context, because I knew I had two toolkits which I could take advantage in this language:
- SlimDX framework: A thin, managed code wrapper around the various DirectX APIs including DirectInput.
- Diffusion .NET Unified Client: A managed code SDK implementing the Unified API introduced with Diffusion 5.0.
There’s no need for the publisher to have a local user interface so I developed this as a console application. I was able to use Microsoft’s free Visual Studio Express 2013 for Windows Desktop to develop, build and test my solution.
In addition to the Thrustmaster steering device I also tested with a standard joystick. This was a fairly aged Logitech Attack 3 unit. Initially my reason for supporting a joystick in addition to steering wheel was to allow me to test my code when I didn’t have the steering wheel with me. This highlighted a bigger issue involving the design of a publisher of this kind, namely the fact that any mapping or normalisation (i.e. physical input to logical publication) should take place at the publisher and not within the client. This keeps the clients simpler and standardises the protocol being created and implemented by way of the topic tree published via Diffusion.
A benefit of using the new Unified API is that my publisher doesn’t need to be installed within the Diffusion server. For testing I ran Diffusion on the same Windows machine I was developing on (simply a matter of locating my Diffusion 5.1 installation’s bin folder and running diffusion.bat from there with the default configuration). Eventually, of course, I would expect a live installation to stream out to a Diffusion server hosted where it needs to be accessible to all clients (e.g. the Internet).
My topic tree root is F1Publisher/ with the following sub-topics:
- Acceleration (Single): In the range 0.0 (foot off pedal) to 1.0 (‘pedal to the metal’).
- Braking (Single): In the range 0.0 (foot off pedal) to 1.0 (full braking force).
- Names (Record): Strings. Where each field represents the name of a button. Fields are in the same order as for States.
- States (Record): Booleans. Where each field represents the state of a button with a value of 0 indicating Off (not pressed) and 1 indicating On (pressed).
- Gear (Single): In the range 1 to 8 to represent the range found in a typical F1 car.
- CountOfFailedTopicSourceUpdates (Single): Unsigned integer.
- CountOfSuccessfulTopicSourceUpdates (Single): Unsigned integer.
- CountOfUpdates (Single): Unsigned integer. Increases by one for each poll of the DirectInput device.
- RateOfSuccessfulTopicSourceUpdatesPerSecond (Single): Unsigned integer. Updated by the publisher once a second. Used to monitor the rate of Diffusion topic source updates.
- RateOfUpdatesPerSecond (Single): Unsigned integer. Updated by the publisher once a second. Used to monitor whether the desired rate of input device polling updates is being achieved.
- UpTimeInSeconds (Single): Unsigned integer. How long the publisher’s been running.
- RefreshInterval (Record): Two fields, both unsigned integers. The first field contains the currently selected input device polling frequency in hertz (updates per second). The second field contains the applied ‘sleep duration’ between updates in milliseconds. These two values are inextricably linked but I decided to transmit them separately (albeit atomically, being a single record). This ensures that the client doesn’t need to duplicate the maths involved in translating between one and the other (open to rounding errors, etc..).
- Steering (Single): I nominated a half turn of the steering wheel as one unit with a turn to the left being negative and a turn to the right being positive.
I decided to experiment with the record metadata feature that was added to the .NET unified client for version 5.1. I used it to name the fields containing button states with their corresponding button names. This was more of an academic exercise as the iOS client library does not currently support this feature. To provide the iOS client with this information I worked around this by adding an additional record data topic where the fields contained the names of the buttons in the same order as they were presented in the states topic.
Output: The Display Client
As a real world application, it’s likely that it would be convenient for users of the display client to run it on their phone or tablet. Perhaps as a second screen device.
I spent a number of years developing mobile applications for Apple iOS devices prior to joining Push Technology. This means that I’ve had immediate and recent experience with Objective-C, Cocoa Touch and Xcode. For this reason, I chose to write my display client in Objective-C for iPhone, iPad and iPod touch.
Naturally, Push Technology also have an SDK for that so I was able to use our iOS client library. This library, at the time of writing, only supports Diffusion’s Classic API. This is perfectly adequate as it provides all the functionality required to consume data from my publisher.
Initially I experienced a little more latency in my display output than I was expecting. I established that this was due to the default, implicit animation applied by Apple’s Core Animation framework when setting animatable properties (e.g. frame and transform) on a layer. This was easily disabled using a CATransaction, setting the animationDuration property to zero.
Running the Demo
Instructions for getting the demo up and running are in the README file located at the root of the repository which is hosted on GitHub:
It’s possible to run the entire demo (publisher, Diffusion and display client) from a single OS X machine with free software alone. In my case this was a mid-2010 15” MacBook Pro with 8GB running OS X 10.9.4 and the following installed:
- Xcode 5.1.1 to build and run the display client using the simulator.
- Oracle VirtualBox VM version 4.3.14 running one of the Windows 7 ‘test’ virtual machine images available from Microsoft at Modern.IE
- Visual Studio Express 2013 for Windows Desktop.
The only tricky elements involved virtual machine configuration:
- Network: I configured my virtual machine’s adapter as attached to ‘Bridged Adapter’ so that it could participate in the same LAN environment that my Mac was on (i.e. grab an additional address via DHCP).
- Ports – USB: I had to add a device filter for my test devices (steering wheel and joystick). Even after adding this filter and rebooting the virtual machine I found that it was still necessary to physically unplug the USB cable and the replug it in order for VirtualBox to recognise it and allow it to communicate with Windows.
The experience of producing this demo was thoroughly enjoyable. It allowed me to rapidly learn while producing immediate and incrementally improving output. Every test run was a tactile experience. I got to spend time renewing my relationship with Microsoft .NET and C# (it had been a few years). And, of course, I got to run Diffusion through its paces… It certainly didn’t let me down.
As you might imagine, my colleagues were a little bemused as to why I had a Ferrari-branded steering wheel clamped to my desk with matching pedals at my feet. Perhaps, given my ‘new starter’ status, they were happy to rationalise by assuming I was just an avid gamer who liked to let his hair down during lunch breaks.
What was particularly satisfying was to see how immediately people ‘got’ the demo once they had seen it in action. As such, I feel, I’ve succeeded in producing a demo with the characteristics I was originally seeking.
Where could we take it next? All suggestions welcome!
The Diffusion Intelligent Data Platform manages, optimizes, and integrates data among devices, systems, and applications. Push Technology pioneered and is the sole provider of real-time delta-data streaming™ technology that powers mission-critical business applications worldwide. Leading brands use Push Technology to fuel revenue growth, customer engagement, and business operations. The products, Diffusion® and Diffusion Cloud™, are available on-premise, in-the-cloud, or in a hybrid configuration, to fit the specific business and infrastructure requirements of the applications operating in today’s mobile obsessed, everything connected world. Learn how Push Technology can reduce infrastructure costs, and increase speed, efficiency, and reliability, of your web, mobile, and IoT application.