Adding Over The Air (OTA) updates is an important factor for IoT applications to succeed long term. OTA is a mechanism to ensure that devices are always up to date with new settings and security fixes. It's also useful for adding new features to the hardware, making the customer who bought the device feel happier and more confident with the hardware improvements.

There are two important parts on an OTA architecture:

• The remote device: responsible for checking for updates, downloading the new version, and applying that new version to itself
• The cloud server: responsible for building, distributing, and managing those connected device updates

Here I’ll show how to setup an initial OTA mechanism using Google Cloud tools. Then, I will guide you through the process of deploying those updates to ESP8266 and ESP32 boards using the Arduino platform.

‍PlatformIO will be used for building the images. It has a set of command line tools that enable us to automate the process of generating binary images for the devices. On Google Cloud, we're going to useGoogle Cloud Build—a managed Continuous Integration environment—Google Cloud Storage for storing the binary images on the cloud, and Cloud Functions to handle HTTP request querying for current firmware versions and also for managing them.

Getting Started with PlatformIO

PlatformIO is a set of cross-platform tools for developing solutions for embedded devices. It supports a lot of different platforms and frameworks for IoT development. It also has a huge set of libraries made by the community that can be easily used in your project. I recommend installing the Visual Studio Code (VSCode) IDE and the PlatformIO plugin to get started. Follow the steps below below:

The code for this project is available here on Github. Clone or download the project and open it in your IDE.

The platformio.ini file contains all the configuration to build the project on the ESP32 and ESP8266 boards. Also, the project dependencies are listed here. An important configuration is the build flag VERSION, which is compiled on the project code to mark which version the device is currently running. So, every time we create a new firmware version, this code should be bumped so the device will be able to check whether it needs to update to a new version.

Platformio.ini Gist Link

The device code makes an HTTP query to the backend, sends the current version, and checks whether it should download a new one. Additionally, there's a device-internal HTTP handler to display the current version. To handle WiFi connectivity, the project uses the WiFiManager library, which creates an access point to setup WiFi on the device.

Main.cpp Gist Link

To deploy to the board, you can use the “Build” and “Upload” buttons on the PlatformIO Toolbar:

Setting Up Cloud Build and Git Repositories

To get started with Google Cloud, you can do everything on the Cloud Console web interface. The command line tools are, however, a more powerful toolset. Later, we’ll need to deploy our cloud function.

• Follow the instructions here to download and install the Google Cloud command line tools
• Read this guide and implement the steps for installing and setting up the Google Cloud SDK
• Follow these steps to create a new Google Cloud project. You should authenticate and create a project to use in this tutorial using the format YOUR_PROJECT_NAME

——————————————————————————

# Authenticate with Google Cloud:
gcloud auth login
# Create cloud project — choose your unique project name:
gcloud projects create YOUR_PROJECT_NAME
# Set current project
gcloud config set project YOUR_PROJECT_NAME

——————————————————————————

Now let’s create the Cloud Build configuration and also the Bucket in which we can store the binaries. Follow these steps:

Cloud Build Setup

• Open the Cloud Build page
• Enable Cloud Build API if it has't yet been enabled
• Select the Triggers tab
• Click on Create Trigger
• Now you can select your Git repository and authenticate with your provider (I used Github)
• On the last step, Trigger Settings, let’s create a trigger by git tags
• Name it — “Trigger by Build Tag”
• Select Tag as the Trigger type
• Select cloudbuild.yaml as the Build Configuration type
• Click on Create Trigger

Cloud Storage Setup

• Now open the Cloud Storage page
• Select the Browser tab
• Click on Create Bucket
• Create a bucket with the name YOUR_PROJECT_NAME-firmwares and have it select the other values by default (the name of my bucket was 'gcloud-ota-update-firmwares')
• Click on Create

Our repository contains a cloudbuild.yaml file, which contains all the configuration to build the firmware and push it to Cloud Storage. Cloud Build uses Docker for building artifacts, so I used an image that contains all the PlatformIO tools for building embedded projects using our platformio.ini file.

Cloudbuild.yaml Gist Link

Now, every time that you push a new tag to your repository, it'll trigger a build on Cloud Build. You can create the tag on the UI of your git provider or you can do so using the following git commands:

——————————————

git tag -a v1.0.0 -m "First build"
git push -u origin --tags

——————————————

And if everything is working correctly up to this point, you should start seeing some builds on the History tag of your Cloud Build page when you push a new tag. We’ll revisit this at the end of the post to see how to push new versions.

Deploy Cloud Functions

To control the OTA process, we need two things:

• Store firmware metadata in a database so that we can query later for the latest version
• A way to check whether given the current device version, a device update is needed

I've built two cloud functions that help us achieve these two requirements:

• insertFirmwaresOnBigquery: This function is triggered when new files are uploaded to Cloud Storage. We store the metadata on BigQuery so that we can query and filter it later by device variant and firmware version. insertFirmwaresOnBigquery.js Gist Link‍
• getDownloadUrl: This is an HTTP function that receives the current device version and also its variant (in this case, esp32 or esp8266).The function then queries BigQuery for the latest version and compares the device version with the latest firmware version. getDownloadUrl Gist Link

Now, you'll need the gcloudtool that we installed in the beginning to deploy the functions. Alter the project ID on the file deploy-prod.sh and run it to deploy both functions. The first time you run the functions, you'll probably be asked to enable the cloud functions API. Just confirm and continue with the process.

————————

./deploy-prod.sh

————————

With this command, all the functions are deploying and reacting to events in our architecture.

Pushing a New Version

To push and build a new firmware version that can be download by the device is simple. With just a couple of git commands, we can trigger a new Continuous Integration build. Let’s say that you add a new feature to the device, e.g. a blink on our loop function.

Blink.cpp Gist Link

Now to create a new release, change the version on platformio.ini file (from v1.1.0 to v1.2.0 for example), commit the changed files, tag the new commit, and push everything to the repository. Here are the commands:

————————————————

# Commit the files
git add src/main.cpp platformio.ini
git commit -m "[feat] Add blink feature"
# Tag this commit
git tag -a v1.2.0 -m "Add blink feature"
# Send to the repository with the tags
git push -u origin master --tags

————————————————

‍

Conclusion

I hope that this tutorial gave you an overview of what can be done to automate the firmware update process in order to achieve better IoT deployments. We went through many of the components involved in an OTA deployment process, but our architecture is still too simple. We can improve it by adding many more features, such as:

• Release channels: Have different devices check for different channels, e.g. production, beta, and alpha channels. In this way, a solutions provider can restrict their system such that only internal devices receive new builds to be tested by the company and then push the tested version to all of their customers.
• Private keys: Use private keys on devices to validate whether the device can receive those updates. This would make the whole process more secure.
• Check for updates within a specified time frame and provide user feedback: Should the customer be using the device when the update arrives, we should handle this and wait for user confirmation before going through the update process. This way we don’t block user interaction with the device just because a new update arrived.