Realtime telemetry with Nexmo and Ably

In this article I take a quick look at building a telemetry system with Nexmo and Ably. The idea is that remote sensors, which could be measuring temperature, smoke, river levels, pollution, light levels, or anything else, periodically send in an SMS with the sensor data. This is especially useful where sensors are not in a WiFi zone, but there is mobile phone coverage. The hardware is based around cheap devices such as Arduinos or Raspberry Piis, with an SMS module, but I don't go into hardware in this article. In my basic example I host the SMS server on Glitch and feed the data into Ably.

With Ably you can have a "firehose" of data from potentially thousands on input sensors, and have numerous clients consuming the data on the channels to which they subscribe, in realtime. This saves you having to make expensive REST API calls from a client, as we did in the FlySpy project. Using Ably allows your system to scale to many thousands of sensors and consuming clients.

For the server in this case I eschew my usual Python and go for a Node/Express combo - basically because Ably currently doesn't have a realtime Python SDK from what I can make out, and I wanted to put everything on a single server just for ease of trying things out. Glitch allows you to create a new Node/Express project with a single click. Did I mention I love Glitch? ;)

What you'll need

In this example I'll be hosting my webhook server on Glitch. You can get a free account over at Glitch. Hosting web apps on Glitch has proven to be a wonderful experience and I heartily recommend it!

You can also get a free account over at Ably.

A free Nexmo account with two Euros credit can be had at Nexmo.

Nexmo webhook server

In previous articles I have looked at creating Nexmo apps. Things have changed since I wrote those articles, but many of the principles are the same. The best thing to do is sign up for a Nexmo account and have a play. In this case you can set up everything you need to in the Nexmo Dashboard:

Purchase a Nexmo number
Set your SMS webhook

Check out the docs at the developer portal and my previous articles.

For inbound SMS you need to set up an application-level webhook URL in your Settings in the Dashboard.

Once you have created your Glitch account and created a Glitch Node/Express project, you can go into your Nexmo Dashboard and into settings and set your webhook for inbound SMS. For my project the webhook URL is https://flawless-buttery-legal/webhooks/inbound-sms. The code for the webhook server is:

const express = require("express");
const bodyParser = require('body-parser')

var key = process.env.API_KEY
var secret = process.env.API_SECRET

var ably = new require('ably').Realtime(key + ':' + secret);
var channel = ably.channels.get('my-telemetry');

const app = express();

app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: true }))

// For the JavaScript Ably client
app.get("/", (req, res) => {
  res.sendFile(__dirname + "/views/index.html");
});

app
  .route('/webhooks/inbound-sms')
  .get(handleInboundSms)
  .post(handleInboundSms)

// Ably Node client
channel.subscribe('temp-sensor', function(message) {
  console.log('Temp sensor: --> ' + message.data + ' Centigrade');
  console.log('=========')
});

// Inbound SMS webhook handler
function handleInboundSms(req, res) {
  console.log('Inbound telemetry...')
  channel.publish('temp-sensor', req.body['text']);
  res.status(204).send()
}

// listen for requests :)
const listener = app.listen(process.env.PORT, () => {
  console.log("Your app is listening on port " + listener.address().port);
});

p.s. sorry about being really inconsistent with semi-colons. Use Prettifier if you must! ;)

Setting up an Ably channel

The code to set up your Ably channel is:

var ably = new require('ably').Realtime(key + ':' + secret);
var channel = ably.channels.get('my-telemetry');

Inbound SMS webhook handler

The inbound SMS webhook handler is called every time Nexmo receives an inbound SMS on your Nexmo number. In this case, you simply publish the data to an Ably channel. The code is:

function handleInboundSms(req, res) {
  console.log('Inbound telemetry...')
  channel.publish('temp-sensor', req.body['text']);
  res.status(204).send()
}

The Client consuming data on the Ably channel

Clients subscribed to the my-telemetry channel can then read messages of the required type when they come in, in realtime. This is done with the code:

channel.subscribe('temp-sensor', function(message) {
  console.log('Temp sensor: --> ' + message.data + ' Centigrade');
  console.log('=========')
});

Here you can see the client subscribes to the channel and in particular messages of type temp-sensor. The subscribe method provides a callback, which is called when a message of type temp-sensor comes in. In this case you just log the data to the console, but you could do much fancier things, such as display the data on a chart, or store it in a database such as Couchbase.

Another Ably client

In this rather contrived example the client consuming data in realtime via Ably is in the webhook server, but it needn't be. For example, I also added a browser client. The code is:

<!DOCTYPE html>
<html lang="en">
  <head>
    <script src="//cdn.ably.io/lib/ably.min-1.js"></script>
  </head>
  <body>
    <h1>Ably test</h1>
    <p>Hello</p>
    <script>
      var ably = new Ably.Realtime(
        "YOUR_ABLY_JWT"
      );
      ably.connection.on("connected", function() {
        console.log(
          "That was simple, you're now connected to Ably in realtime"
        );
      });
      var channel = ably.channels.get("my-telemetry");
      channel.subscribe("temp-sensor", function(message) {
        alert("Received a message in realtime from temp sensor: " + message.data);
      });
    </script>
  </body>
</html>

Here the Ably JavaScript SDK is used to connect to Ably. The code is almost identical to that shown already. I use the Chrome developer tools to view the JavaScript console. Of course I could actually display the inbound data in the browser, but didn't get around to it, so I leave that as an exercise for the reader! ;)

The Ably JWT

You'll notice I used an Ably JWT to authenticate. For Ably that is one of several authentication schemes you can use. Of course, in a client you do not want to expose your secret key and password so that option is out here. You would probably have your client authenticate using, for example, a login box, which is then authenticated by your authentication server. Ably allows you to provide a callback which is called whenever a new JWT is required for authentication. This is all covered in detail in the Ably docs.

Generating an Ably JWT

For testing purposes I whipped up my own Ably JWT generator in Python (and then did a JavaScript version for fits and giggles). Here's the Python code:

#!/usr/bin/env python3
import os
import jwt
import time
from uuid import uuid4
from dotenv import load_dotenv

load_dotenv()
kid = os.getenv("KID")
secret = os.getenv("SECRET")
exp = os.getenv("EXP")
acl = os.getenv("ACL")

payload = {}
payload['iat'] = int(time.time())
payload['exp'] = int(time.time()) + int(exp)
payload['x-ably-capability'] = acl

token = jwt.encode(payload, secret, algorithm='HS256', headers={'kid': kid})

j = token.decode(encoding='ascii') # Convert byte string to printable string
print(j)

This is very similar to code I have described in the past. Please see my old articles on understanding JWTs and writing your own JWT generator.

You could also do something fancier, with a UI and just run it in the browser. Here's the code:

<html>

<head>
  <script type="text/javascript" src="/bower_components/crypto-js/crypto-js.js"></script>
  <script src="/script.js"></script>
</head>

<body>
  <h1>Ably JWT generator</h1>

  <label for="kid">kid:</label><br />
  <input type="text" id="kid" /><br />

  <label for="secret">secret:</label><br />
  <input type="password" id="secret" name="secret" /><br />

  <label for="exp">expiry:</label><br />
  <input type="text" id="exp" name="exp" /><br />

  <label for="acl">acl:</label><br />
  <input type="text" id="acl" name="acl" /><br />

  <button onclick="genAblyJWT()">Gen Ably JWT</button>
  <hr />
  <textarea name="token" id="token" rows="10" cols="30">
      Your JWT will appear here
    </textarea>
</body>

</html>

This code gives you a little UI you can plug values into. There script code is:

function genAblyJWT() {

    var kid = document.getElementById("kid").value;
    var secret = document.getElementById("secret").value;
    var exp = document.getElementById("exp").value;

    var header = {
      "typ": "JWT",
      "alg": "HS256",
      "kid": kid,
    };

    var currentTime = Math.round(Date.now() / 1000);

    var claims = {
      "iat": currentTime /* current time in seconds */,
      "exp": currentTime + 3600 /* time of expiration in seconds */,
      "x-ably-capability": '{"*":["*"]}',
    };

    var base64Header = b64(CryptoJS.enc.Utf8.parse(JSON.stringify(header)));
    var base64Claims = b64(CryptoJS.enc.Utf8.parse(JSON.stringify(claims)));
    var token = base64Header + "." + base64Claims;


    /* Apply the hash specified in the header */
    var signature = b64(CryptoJS.HmacSHA256(token, secret));
    var ablyJwt = token + "." + signature;

    document.getElementById("token").innerHTML = ablyJwt;
  }

  function b64(token) {
    encode = CryptoJS.enc.Base64.stringify(token);
    encode = encode.replace( /\=+$/, '');
    encode = encode.replace(/\+/g, '-');
    encode = encode.replace(/\//g, '_');

    return encode;
  }

You'll have to install CryptoJS using Bower as this is all client-side, and just for producing JWTs locally in the browser.

I was able to authenticate using these JWTS, and also don't forget to check them via this great JWT debugger.

More on Ably

I haven't described many cool features of Ably. For example, you can go into the Ably Developer Console in your Dashboard and investigate the messages on your channel. I hope to be covering a lot on Ably in future articles, including a conversion of FlySpy to Ably, complete with realtime mapping client.

Summary

In this article I gave a quick intro to building your own SMS-based telemetry system using Nexmo and Ably. I have only scratched the surface of what is possible here. I have not gone into too much detail on everything here, as I provided the code, but as always, if you have any questions you can also get in touch. Remember I am no expert on these things, but I do enjoy sharing my own learning experiences. Until next time...

Welcome to Tony's Notebook