Live Streaming — Walkthrough#

This tutorial demonstrates live streaming to ATLAS, using InfluxDB, and building on the architecture from Tutorial 03.

At the end, you'll have ATLAS connected to a popular off-the-shelf timeseries database, showing live updates.

In this tutorial you:

Update the Influx Data Adapter demo to add Redis and the RTA Stream Service
Generate and ingest data into InfluxDB in real-time, while also publishing live updates Redis
Test the deployment with ATLAS

Tip

These code samples are in C# for .NET 5.0, and use McLaren NuGet packages to keep things as simple as possible.

We recommend you follow this tutorial even if you are planning to do your integration in another language.
The concepts should translate very easily once you have a working example.

Prerequisites#

You'll want to complete the Influx Data Adapter Tutorial first.

This should get you setup with everything you need.

Step 1: Stop your existing Docker stack#

Hit Ctrl+C to stop your Docker Compose stack from the previous tutorial.

Step 2: Edit the Docker Compose deployment#

Make sure you're working from the deployment for the Influx Tutorial.

Redis#

Modify the docker-compose.yaml to add Redis as an extra service:

services:
    redis:
        image: redis:6.2
        restart: always
        ports:
        - "6379:6379"

    # ...

Stream Service#

Modify the docker-compose.yaml to add the Stream Service:

services:
    redis:
        image: redis:6.2
        restart: always
        ports:
        - "6379:6379"

    rta-streamsvc:
        image: mclarenapplied/rta-streamsvc:latest
        restart: "always"
        ports:
        - "8180:80"
        environment:
        - RTA_RedisConnectionString=redis

    # ...

Start the Stack#

Start the services from a terminal in the folder where you saved the docker-compose.yaml file:

docker compose up

Step 3: Check the Services#

HTTP#

Using a web browser, make sure all these health endpoints are reporting Healthy:

Service	/health link
rta-gatewaysvc	http://localhost:8080/health
rta-streamsvc	http://localhost:8180/health
rta-sessionsvc	http://localhost:2650/health
rta-configsvc	http://localhost:2660/health
rta-datasvc	http://localhost:2670/health
rta-schemamappingsvc	http://localhost:2680/health
rta-influxdatasvc	http://localhost:2690/health

If a service says it is Unhealthy, the console log output should say why.

gRPC#

The Stream Service does not introduce any new gRPC interfaces to check.

Step 4: Run the Demo#

Open the tutorial solution in your development environment (e.g. Visual Studio Code).

This time, you will be running RTA.Examples.InfluxLive.

From the terminal, navigate to this sub-directory, and then:

dotnet run

If the service has been setup correctly, this will live-stream data into InfluxDB and Redis, where it can be served to ATLAS.

By default, the demo will create a series of five-minute sessions.

Step 5: Setup ATLAS Live Connections#

Setup the live connection#

Open the ATLAS Session Browser.

Right-click on Sources and select Service Connections...

Select the service connection you previously defined, and click Properties to open a dialog.

Fill in the Web Socket URI: ws://localhost:8180/

Close the dialogs, and right-click Sources and select Refresh.

Un-tick and re-tick the localhost source to make sure it picks up the new settings, and press the Refresh button on the right-hand-side of the dialog. You should see your existing sessions, and a live session at the top of the list.

Troubleshooting

If you don't see a live session:

Double-check the Web Socket URI is correct
Open the URI in a web browser the the 'http://' scheme and make sure you are communicating with the Stream Service
Restart ATLAS and try again
Check the ATLAS log file (Tools > View Log Folder), which may explain what the problem is

If you only see live data, with no back-fill to the start of the live session, this means that data is not loading from InfluxDB.

Test the session#

Double-click to load a live session.

Add a Waveform display, and press P to add parameters using the Parameter Browser.
You should see waveform traces updating in real time.

If you unload and reload the session, you should see that all the data is still visible.

Info

You may notice a small discontinuity where the data from the Gateway Service joins the data from the Stream Service.

This is a temporary limitation in the Stream Service implementation that will be fixed in a future release.
Redis should contain enough streamed data to seamlessly cover this gap.

You may also notice that the time-range in the displays extends back beyond the start of the session.
This appears to be a time-zone issue, which will be looked at and fixed in a future development release.

Next Steps#

Review the changes and the sample project to understand how the live stream is fed from the demo process to ATLAS.