Archive

Integration

For some just beginning their programming journeys a common example to conquer is blinking an LED, which usually goes something like this:

digitalWrite(13, HIGH);
delay(1000);
digitalWrite(13, LOW);

For me, I decided to try a much harder approach, in a fiddly effort that could be regarded as virtually pointless. Nevertheless, I persisted, because I thought it would be cool.

The idea: blink a Blinkt LED on Server Pi whenever it serviced a request from the outside.

For those unfamiliar with my little family of Raspberry Pi minions, here is a brief overview:

  • Server Pi – A Raspberry Pi 3 running three Node.js processes for various Pebble apps (News Headlines pin pusher, Tube Status pin pusher, unreleased notification and discovery service).
  • Backlight Pi – Another Raspberry Pi 3 with a single Node.js Express server that allows any device in the house to HTTP POST a colour to be shown behind my PC.
  • Monitor Pi – A Raspberry Pi Zero W (W, as of today) that pings the three processes running on Server Pi via the GitHub Gist discovery mechanism to give me peace of mind that they’re still up. It also checks the weather for ice and rain, and whether or not Greater Anglia have fallen over before I’ve taken the trouble of leaving for work at 7AM.

Maintaining this small fleet is a joy and a curse (one or both of “my own mini infrastructure, yay!” or  “It’s all fallen over because Node exceptions are weird, noo!”), but since I started versioning it all in Git and adding crontab and boot scripts, it’s become a lot easier. However, for this particular task, I found only one process can usefully control the Blinkt LEDs on top of Server Pi. Since this is a parameterised (services only) instance of Monitor Pi, it must be this process that does the blinking when a request is processed.

Since I’m already a big fan of modular Node.js apps, I just added another module that sets up a single-endpoint Express server, and have each of the other three Server Pi processes POST to it whenever they service a request with their own Express servers. Neat!

An hour of synchronising and testing four processes locally and on-device later, and I now have a blue blinking LED whenever a request is serviced. Sadly the activity isn’t as high as it was in the News Headlines heyday when it was tasked with converting news story images to Pebble-friendly 64 colour thumbnails and an experimental analytics service late last year, but with the interesting tentative steps the unreleased notification service is taking, Server Pi may end up seeing a bit more action than simple status checks and app news lookups in the future.

With all this work done, it’s also time for another diagrammatic mess that I like to call my infrastructure…

This one took a while. Weighing in at 38 versions and I-don’t-know-how-many reflected and hacked APIs, Dashboard is now open-source for all to see. This was probably the app that took the most development time until now (still on-going!), and I think the one I’m most proud of.

I’m liking the idea of doing future releases via self-approved pull requests. Could be interesting!

>>> Dashboard source code <<< 

Here’s a little something to take everybody’s mind of things.

 

This is a neat idea I had a while ago but only just got around to doing –

“What would a map of all the interconnections and services that my Pebble apps rely upon look like?”

Well, thanks to the neat tool that is Google Drawings, here it is. Scary dependencies!

services-architecture

Of course, this isn’t the full picture. The Server Pi and Monitor Pi provide me with useful services I use in my day-to-day life, such as train delay timeline pins, weather alerts, and updates on the health of the services apps rely upon. Those details aren’t shown here for brevity, but would increase the complexity of the drawing about 50%.

I’m a big fan of Blinkt light hats for Raspberry Pi. I have one showing me server status, rail delays, and weather conditions.

_20161016_153730

Server down!

I have another at work showing the status of the last link check on our ReadMe.io site.

img_20161117_165458

And now I have one as a dynamic backlight for my new PC build.

img_20161120_135026

And the best part? This last one has an API! It has five modes, powered by a Node.js Express server and the node-blinkt NPM package:

  • /set { "to": [r, g, b] } – Set a color instantly.
  • /fade { "to": [r, g, b] } – Fade from the current colour to this one.
  • ‘CPU’ – Fade to a colour on a HSV scale from green to red depending on current CPU load.
  • ‘Screen’ – Take an average of the four screen corners, and set to that colour ten times a second.
  • ‘Demo’ – Fade to a random colour from the rainbow every 15 seconds.
  • ‘Test’ – Ping the Pi and set the ‘Test’ button to green if it responds ‘OK’ and HTTP 200.

The last three are driven by a Java control panel that permanently lives on the new PC.

controller

Thanks to the motherboard supplying power after the PC turns off, I can use ‘Demo’ as a nightlight! Not that I need one…

Update: Added changed IP facility details.

Update: Added status watchapp details.

Two of my Pebble apps push pins to the timeline to enhance their experience beyond the apps themselves:

  • News Headlines – Posts the top headline (if it’s new) every four hours. Used to push notifications and serve decoded PNG images, but that went away. Maybe someday they will return. But not for now.
  • Tube Status – Checks the TFL feed every five minutes, and pushes a pin if there is a change in the delay status. This can be a new delay, a delay that’s ended, and ‘all clear’ (no delays anymore).

Both servers also respond to GET /status to show app users if they’re up, and this has proved useful when they occasionally went down. Thanks for a ‘do node index.js forever’ loop script, this is very rarely now an issue.

Up until now, these pins were served from a $5 Digital Ocean instance which essentially spends 99.9% of its time doing absolutely nothing! After coming back to the UK and making progress towards cancelling subscriptions and emptying my US bank account, I had a better idea – use my dusty Raspberry Pi instead!

As part of my new job at EVRYTHNG, a natural avenue of exploration for getting to grips with the IoT is using a Raspberry Pi, which can run Node.js, as it turns out. Perfect! The pin servers for the two apps above use Node.js with Express.

So after a bit of code/dependency cleanup, I set up both servers on the Pi with screen and put plenty of stickers around warning against turning it off or rebooting the router.

So far, so good! What could go wrong?

img_20160911_143438

The new ‘Pin Pusher’ Raspberry Pi in its native habitat – under the family computer desk.

Followup: Getting a Changed Router IP while Out the House

In the eventuality that I have to update the IP of the family router for apps to use in their status check (otherwise they think the servers have gone down, bad for users!), I used to have a Python script email me its own IP address. Sadly, Google doesn’t like this unauthenticated use of my GMail account, so I devised an alternative.

I set up my Pi as an EVRYTHNG Thng, gave it an ‘ip’ property, and wrote the following Python script to update this property in the EVRYTHNG cloud when it boots up. This way, all I have to do is ask whoever’s in to reboot the Pi, and then wait for the updated IP address! I may also make it run periodically to cover the ‘router randomly restarted’ scenario.


#!/usr/bin/python

import requests
import socket
import fcntl
import struct
import json

user_api_key = "<key>" # Probably shouldn't publish this!
thng_id = "<id>"

def get_ip_address(ifname):
  r = requests.get("http://www.canyouseeme.org")
  spool = r.text
  start_str = "name=\"IP\" value=\""
  start_index = spool.index(start_str) + len(start_str)
  spool = spool[start_index:]
  end_index = spool.index("/>") - 1
  return spool[:end_index]

def main():
  ip = get_ip_address("eth0")
  print("IP: {}".format(ip))

  headers = {
    "Authorization": user_api_key,
    "Content-Type": "application/json",
    "Accept": "application/json"
  }
  payload = [{
    "value": ip
  }]
  r = requests.put("https://api.evrythng.com/thngs/{}/properties/ip".format(thng_id), headers=headers, data=json.dumps(payload))
  res = r.text
  print(res)

main()

Followup: Checking Status Conveniently

Each of the two apps mentioned above have a built-in server monitoring feature in their settings screens, but that’s a lot of scrolling. To put my mind at ease I have also created a simple monitoring app that uses the same backend mechanism:

img_20160911_225459

A few days ago, an interesting idea came up in the PebbleDev Slack channel: could a library make it easier for Pebble developers to integrate their apps with Android APIs, but save the pain of each and every one of them needing to publish an app with PebbleKit Android? This would be similar to how Dashboard operates, but by sharing the API access to other apps that are installed.

Turns out, it can! In one of my signature coding bursts I set to work, and prototyped a system that did just this. The result is the Dash API. With it, you can finally (and easily) provide one of the most common watchface widget requests – things like the phone’s battery level, or connectivity status.

How does it work?

The Dash API (named after Dashboard) Pebble package uses AppMessage to send requests to read data (such as WiFi network name, or phone battery level), or write to an API (such as turning off WiFi). Using a unique key to recognise Dash API messages, the Android app can respond to these by reading the data or manipulating the Android API and returning a response code. All the developer needs to do is instruct their users to install my Dash API Android app, and then their app (and any others that use the Dash API) can take advantage of the APIs presented through the library. This approach makes the Dash API an install once, use in many apps service.

The upshot of all this is that a C developer making a watchface that wants to show the phone battery level (such as the demo app Dual Gauge) need only use the Pebble package and not write a single line of Java, let alone go to the trouble of publishing an Android app on Google Play.

But how can one companion app target them all, without knowing they exist?

Luckily, you can extract the UUID of an app that sends an AppMessage packet to the phone from the Intent object broadcast from the Pebble Android app. Most (hopefully app) companion apps that manually register a BroadcastReceiver do a ‘good citizen’ UUID check to make sure they only respond to messages from their corresponding watchapp. This data can be used to simply redirect an incoming message’s result right back at it, without the need to create a PebbleKit Android app with baked in UUID for each individual app.

So how can I use it?

Simple. As explained in the GitHub README.md file, a C developer should first install the package:


$ pebble package install pebble-dash-api

Next, include the single library file and call the initialiser when your app is starting, supplying the app’s name (for presentation in the Android app) and an error callback for receiving any errors that may occur:

#include <pebble-dash-api/pebble-dash-api.h>;
#include <pebble-events/pebble-events.h>

#define APP_NAME "My App"

static void error_callback(ErrorCode code) {
  // Receive error codes here
}

static void init() {
  dash_api_init(APP_NAME, error_callback);
  events_app_message_open();

  /* Other init code */
}

Next, check the Android app is available and up to date:

dash_api_check_is_available();

The result will be passed to your error_callback. Once you get ErrorCodeSuccess, it is safe to start making queries, such as getting the battery level, etc. Code examples are included in the GitHub README.md file.

What else do I need to know?

As of 1.1 (released today), apps that read data and API states can do so invisibly. However, apps that wish to write to an API (such as turning off WiFi) will cause a notification to appear that will prompt the user to grant it access within the Dash API app. Once this is done, operation will continue without further intervention, unless permission is revoked. This should hopefully prevent any rogue apps messing with the phone, and give users visibility into which Pebble apps they have used are using the Dash API.

Links

GitHub Repo

Dash API Android app (direct your users here!)

NPM Package

Example App (Dual Gauge)

What’s next?

In the future, I’d like to expand the capabilities offered by the Dash API to other popular widget requests, things such as unread SMS count and next Calendar appointment. Perhaps for another day – coordinating GitHub repos, NPM packages, Android apps, and Pebble apps is quite tiring!

That’s right, the latest and greatest update to Dashboard is now live (pending Google Play deployment delay). The new features are (in descending order of number of requests received):

  • A choice of Find Phone sound from system notification sounds. While these are guaranteed to be available, free-form choice of file proved too tricky to handle (see Android file manage content:// URI problems discussed elsewhere).
  • An option to post an Android notification (and by extension, to the watch) when the phone has fully charged (see details below). This approach is better than the original one planned which involved custom UI in the watchapp, but would rely on an assumption about how long PebbleKit Android would take to launch the watchapp before messaging it.
  • Holding down the button now uses repeat button clicks to reduce wear on the Pebble buttons while scrolling through toggles.
  • Since I’m making bigger use of Notifications, installed a proper white notification icon to improve upon the old ‘white circle’.
  • More misc UI improvements in both app components, including a donation link for those inclined, after some minor success in News Headlines.

So how does the new ‘full charge notification’ feature work? I’m glad you asked!

The Short Answer

Once your phone has reached 100% charge (when it was previously 99% or less), an Android notification is posted, which is displayed on Pebble (for free!) unless for some reason Dashboard was disabled in the Pebble app.

The Long Answer

Oh boy, this was a tricky one. While the ACTION_BATTERY_CHANGED Intent is available for apps to know when the battery level changes, it’s a special case that cannot be simply registered in AndroidManifest.xml for spin-up whenever required. It has to be programmatically registered. For an app designed to run with extremely minimal interaction with the Android app component, this is tricky.

So what have I done? When the user toggles the feature on in the Android app, the BroadcastReceiver for the above Intent is registered. Sounds simple, right? Well that works for anywhere between ten minutes and an hour or so. I was charging my phone from 99% in testing, and seeing the notification after a few minutes as expected, but a frustrating lack of notification after leaving the phone idle for couple of hours on longer charges.

For reasons unknown to me, the system kills my programmatically registered BroadcastReceiver at seemingly random intervals. Remember, this wouldn’t happen with a nice and convenient receiver registered in AndroidManifest.xml.

Undeterred by this (because this is a nice feature idea), I found another solution. I made the receiver object a static class member, and used the Android AlarmManager to check it was not null at infrequent intervals (currently half an hour). This seemed to work well, once I realised by trial and error that Android Studio appears to kill BroadcastReceivers and AlarmManager alarms, but not deliver the ACTION_PACKAGE_CHANGED which I use to restore the alarms after a theoretical future app update is installed.

In addition, new users would not have their ‘keep alive’ alarms registered upon first install, since ACTION_PACKAGE_ADDED is not delivered to the app that has just installed (why!?). Fancy another complication? Here you go: AlarmManager alarms are all killed when the device is rebooted, so I make additional use of ACTION_BOOT_COMPLETED to restore them.

Finally

So there we go! I’ve always been a vocal proponent of Android’s Intent system for IPC, but after this set of complications (which has still yielded an apparently solid solution), I’m not so sure. Ah, who am I kidding? Still a big fan!

Dashboard can be installed from the Pebble appstore and the Google Play Store.