Posts Tagged LoRaWAN
AWS IoT Core for LoRaWAN, AWS IoT Analytics, and Amazon QuickSight
Posted by Gary A. Stafford in Analytics, AWS, Cloud, IoT, Python, Software Development on April 15, 2021
In the following post, we will learn how to monitor indoor air quality (IAQ) using a private LoRaWAN sensor device network. The devices transmit their sensor telemetry to AWS through a LoRaWAN gateway using the newly released AWS IoT Core for LoRaWAN service. We will then analyze and visualize the sensor data using AWS IoT Analytics and Amazon QuickSight.

Introduction
On December 15, 2020, AWS announced support for Semtech’s low-power, long-range wide area network (LoRaWAN) connectivity. LoRaWAN devices and gateways can now connect to AWS IoT Core using AWS IoT Core for LoRaWAN. AWS IoT Core for LoRaWAN is a fully managed feature that enables customers to connect wireless devices that use the LoRaWAN protocol with the AWS cloud. Using AWS IoT Core, customers can now set up a private LoRaWAN network by securely connecting their LoRaWAN devices and gateways to the AWS cloud — without developing or operating a LoRaWAN Network Server (LNS).

The LoRa Alliance describes the LoRaWAN specification as a Low Power, Wide Area (LPWA) networking protocol designed to wirelessly connect battery operated ‘things’ to the internet in regional, national, or global networks, and targets key Internet of Things (IoT) requirements such as bi-directional communication, end-to-end security, mobility and localization services. The LoRaWAN specification defines both the device-to-infrastructure (LoRa) physical layer parameters and the (LoRaWAN) protocol, providing seamless interoperability between manufacturers.
According to Wikipedia, LoRa (Long Range) is a proprietary low-power wide-area network modulation technique. It is based on spread spectrum modulation techniques derived from chirp spread spectrum (CSS) technology. It was developed by Cycleo of Grenoble, France, and acquired by Semtech, the founding member of the LoRa Alliance, and it is patented.
AWS-qualified Hardware
Along with the AWS IoT Core for LoRaWAN announcement, AWS released a list of qualified gateways found in the AWS Partner Device Catalog. The catalog helps customers discover qualified hardware that works with AWS services to help build and deliver successful IoT solutions. AWS IoT Core for LoRaWAN supports open-source gateway-LNS protocol software called LoRa Basics Station, an implementation of a LoRa packet forwarder. The AWS IoT Core for LoRaWAN gateway qualification program enables customers to source pre-tested LoRaWAN gateways and developer kits that meet the required LoRa Basics Station specifications.

LoRaWAN Gateway
In this post, we will use the AWS-qualified LoRaWAN-compliant MiniHub Pro gateway by Browan Communications. At $109, MiniHub Pro is a low-cost gateway that uses LoRa Basics Station to forward RF packets received from LoRaWAN devices (uplinks) to the LoRaWAN Network Server (LNS), part of the AWS IoT Core for LoRaWAN service. The MiniHub Pro is based on FreeRTOS, the real-time operating system for microcontrollers.

LoRaWAN Devices
In addition to the gateway, we will use a set of two Model TBHV110 915 Mhz Healthy Home Sensor IAQ (aka Tabs Healthy Home), also by Browan Communications Inc. According to Browan, the Healthy Home Sensor utilizes LoRaWAN connectivity (LoRaWAN 1.0.3) to communicate the temperature, relative humidity, volatile organic compound (VOC) levels, and indoor air quality (IAQ) of the surrounding environment. The Healthy Home Sensor costs $60 per sensor. The gateway and two devices used in this post were purchased from Cal-Chip Connected Devices at a total retail cost of $229 plus tax and shipping.

IAQ Index
The Healthy Home Sensor determines an IAQ Index, a reading between 0–500, indicating general air quality. According to the CDC, monitoring and maintaining good indoor air quality and proper ventilation have become essential to reduce the potential airborne spread of COVID-19.

Source Code
The source code for this post is available on GitHub. Use the following command to git clone
a local copy of the project.
git clone --branch main --single-branch --depth 1 \
https://github.com/garystafford/lorawan-iot-core-demo.git
The Lambda function and its associated AWS resources are deployed using the AWS Serverless Application Model (SAM). The primary AWS resources used in this demonstration are shown in the architectural diagram below.

Adding Gateways and Devices
In combination with the manufacturer’s product manual, the AWS documentation walks you through adding your LoRaWAN gateways and devices with AWS IoT Core for LoRaWAN. Adding gateways and devices is easy and straightforward on AWS as long as you have the appropriate GatewayEUI, DevEUI, AppEUI (aka JoinEUI), and AppKey, supplied by the manufacturer or retailer.

Over-the-Air Activation (OTAA)
In this post, the gateway and sensor devices are connected to AWS IoT Core for LoRaWAN using Over-the-Air Activation (OTAA). According to both The Things Network and AWS, OTAA is preferred over Activation by Personalization (ABP) as it is more secure. When using OTAA, temporary session keys are derived from root keys on each activation. ABP uses static keys, is less secure, and should not be used if not explicitly required by the use case.

The LoRaWAN gateway is added to AWS IoT Core for LoRaWAN. The gateway is then configured locally using the information received from IoT Core. With the MiniHub Pro, you can enable FreeRTOS Over-the-Air Updates and Configuration and Update Server (CUPS).


Below, we see that the MiniHub Pro gateway has been successfully added to IoT Core and has exchanged uplink frames (RF packets) through that connection with the AWS IoT Core for LoRaWAN, which is acting as the LoRaWAN Network Server (LNS).

We then add the two Healthy Home Sensor devices in a similar fashion.

Below, we see that one of the two Healthy Home Sensor devices has successfully connected to AWS IoT Core for LoRaWAN and has also transmitted uplink frames via the MiniHub Pro gateway.

IoT Destinations
Messages from the devices are received from the gateway and routed to an IoT rule using a Destination. According to AWS, a destination describes the AWS IoT rule that processes the data from a wireless device so that AWS IoT services can use the data. Many devices can share a single destination.

IoT Rules
IoT rules, according to AWS, tell AWS IoT what to do when it receives messages from your devices. Rules extract data from messages, evaluate expressions using message data, and invoke one or more actions when the rule’s conditions are met.

Using AWS IoT Core for LoRaWAN, the device’s messages are base64 encoded binary messages. The IoT rule’s query statement is a SQL statement that determines which messages are forwarded to Actions. In the case of LoRaWAN, the query statement contains an inline call to an AWS Lambda function. The function accepts a number of input parameters. It decodes the base64 encoded message, decodes and translates the binary message, and builds a Python dictionary with the results. Finally, the dictionary is serialized to JSON and returned to the IoT rule.
Healthy Home Sensor Messages
LoRa uses a license-free sub-gigahertz radio frequency of 915 MHz in North America. The Healthy Home Sensor device transmits binary messages over this radio frequency using the LoRaWAN 1.0.3 specification. Binary messages sent by the Healthy Home Sensor device to the MiniHub Pro gateway have a payload length of 11 bytes, as follows:

The device encrypts its binary message, containing sensor data, using AES128 CTR mode before transmitting it. AWS IoT Core for LoRaWAN decrypts the binary message, then encodes the decrypted binary message payload as a base64 string.
The final JSON-format message delivered by AWS IoT Core for LoRaWAN the IoT rule contains the device’s base64 encoded binary message (PayloadData), along with additional information about the source LoRaWAN device and the LoRaWAN gateway that transmitted the message.
The IoT rule’s query statement calls a Lambda function to decode and transform the base64 encoded binary message. The Lambda calls the correct decoder, which contains logic to decode each byte of the binary message. AWS has developed the IoT Rule decoder Lambda along with several binary message payload decoders, freely available on GitHub, including:
- Browan Tabs Object Locator
- Dragino LHT65, LGT92, LSE01, LBT1, LDS01
- Axioma W1
- Elsys
- Globalsat LT-100
- NAS Pulse Reader UM3080
Since ASW did not include the Browan Healthy Home Sensor device in the list of provided decoders, I have created a new Python-based decoder using the message payload information detailed in the device’s product manual.
The decoder first decodes the base64 encoded binary message payload.
# base64 encoded binary message
AAs0LNsCAQB/ADM=
# base64 decoded 11-byte binary message
00000000 00001011 00110100 00101100 11011011 00000010 00000001 00000000 01111111 00000000 00110011
# as hexadecimal
00 0b 34 2c db 02 01 00 7f 00 33
# as decimal
0 11 52 44 219 2 1 0 127 0 51
The decoder then processes each byte of the binary message payload, bit-by-bit, and applies any additional logic to derive the final sensor values. Decoding the base64 encoded binary message payload and placing it back into the original message, we are left with the following message structure:
IoT Rule Actions
Messages returned by the rule’s query and processed by the Lambda function are passed to an AWS IoT rule action. AWS IoT rule actions specify what to do when a rule is triggered. This rule’s action sends a message to an AWS IoT Analytics channel.

AWS IoT Analytics
IoT Analytics, according to AWS, is a fully-managed service that makes it easy to run and operationalize sophisticated analytics on massive volumes of IoT data. IoT Analytics consists of five primary components: channels, pipelines, data stores, datasets, and notebooks.

The IoT rule sends messages to the IoT Analytics channel. The channel publishes the data to a pipeline. The pipeline consumes messages from the channel and enables you to process and filter the messages before storing them in the data store. The data store receives and stores the messages. You retrieve data from a data store by creating a SQL dataset or a container dataset. The SQL dataset contains a SQL query, which is executed against the data store. According to the documentation, both Amazon Athena and Amazon IoT Analytics’ SQL expressions are based on PrestoDB.

The SQL query I have written is specific to the data collected by the Healthy Home Sensor device. In Amazon QuickSight, we will use this dataset to construct an IAQ monitoring dashboard.
Amazon QuickSight
Amazon QuickSight, according to AWS, is a scalable, serverless, embeddable, machine learning-powered Business Intelligence (BI) service built for the cloud. QuickSight lets you easily create and publish interactive BI dashboards that include Machine Learning-powered insights. AWS IoT Analytics provides direct integration with Amazon QuickSight.

Using the ‘AWS IoT Analytics’ data source, we can build a QuickSight data set by importing the IoT Analytics dataset built in the previous step. In QuickSight, we are able to preview the data, change field types, apply filters, add calculated fields (e.g., sensor_location
), and exclude fields if desired.

The final data is then saved (cached) in SPICE, QuickSight’s Super-fast, Parallel, In-memory Calculation Engine.https://programmaticponderings.wordpress.com/media/509f9c79155177fd1480eeee2bd23593Final data used for analysis
Using this dataset, we can build a QuickSight Analysis. The analysis will use a variety of visual types to display sensor data, such as temperature, relative humidity, volatile organic compound (VOC) levels, indoor air quality (IAQ), the sensor device’s battery levels, and the gateway’s SNR (Signal-to-Noise Ratio) and RSSI (Received Signal Strength Indication).

We can then securely share the Analysis we have built with data consumers as a QuickSight Dashboard, accessible through a web browser or using the Amazon QuickSight mobile app.

Below, we see how QuickSight is able to visualize indoor air quality data for multiple sensor locations over the course of a single day. In this visual, sensor data is captured and displayed in five-minute increments. The visual contains reference lines indicating both good and bad IAQ thresholds. Analyzing the results, we can quickly see how external and internal environmental conditions, HVAC systems, air filtration devices, external ventilation through windows and doors, and human activity all impact IAQ throughout the day in different areas of the dwelling.

Conclusion
In this post, we learned how easy it is to activate, configure, and start collecting sensor telemetry from LoRaWAN devices and gateways using the newly released AWS IoT Core for LoRaWAN service. We observed the seamless integration of AWS IoT Core, IoT Analytics, and Amazon QuickSight to transform, store, and visualize sensor data. Extending this example to add notifications and auto-remediation based on sensor data is equally as easy with services such as AWS IoT Events.
Reference
Collecting and Analyzing IoT Data in Near Real-Time with AWS IoT, LoRa, and LoRaWAN
Posted by Gary A. Stafford in AWS, Bash Scripting, IoT, Python, Raspberry Pi, Serverless, Software Development on August 26, 2020
Introduction
In a recent post published on ITNEXT, LoRa and LoRaWAN for IoT: Getting Started with LoRa and LoRaWAN Protocols for Low Power, Wide Area Networking of IoT, we explored the use of the LoRa (Long Range) and LoRaWAN protocols to transmit and receive sensor data, over a substantial distance, between an IoT device, containing several embedded sensors, and an IoT gateway. In this post, we will extend that architecture to the Cloud, using AWS IoT, a broad and deep set of IoT services, from the edge to the Cloud. We will securely collect, transmit, and analyze IoT data using the AWS cloud platform.

LoRa and LoRaWAN
According to the LoRa Alliance, Low-Power, Wide-Area Networks (LPWAN) are projected to support a major portion of the billions of devices forecasted for the Internet of Things (IoT). LoRaWAN is designed from the bottom up to optimize LPWANs for battery lifetime, capacity, range, and cost. LoRa and LoRaWAN permit long-range connectivity for IoT devices in different types of industries. According to Wikipedia, LoRaWAN defines the communication protocol and system architecture for the network, while the LoRa physical layer enables the long-range communication link.
AWS IoT
AWS describes AWS IoT as a set of managed services that enable ‘internet-connected devices to connect to the AWS Cloud and lets applications in the cloud interact with internet-connected devices.’ AWS IoT services span three categories: Device Software, Connectivity and Control, and Analytics.
In this post, we will focus on three AWS IOT services, one from each category, including AWS IoT Device SDKs, AWS IoT Core, and AWS IoT Analytics. According to AWS, the AWS IoT Device SDKs include open-source libraries and developer and porting guides with samples to help you build innovative IoT products or solutions on your choice of hardware platforms. AWS IoT Core is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can process and route messages to AWS endpoints and other devices reliably and securely. Finally, AWS IoT Analytics is a fully-managed IoT analytics service, designed specifically for IoT, which collects, pre-processes, enriches, stores, and analyzes IoT device data at scale.
To learn more about AWS IoT, specifically the AWS IoT services we will be exploring within this post, I recommend reading my recent post published on Towards Data Science, Getting Started with IoT Analytics on AWS.
Hardware Selection
In this post, we will use the following hardware.
IoT Device with Embedded Sensors
An Arduino single-board microcontroller will serve as our IoT device. The 3.3V AI-enabled Arduino Nano 33 BLE Sense board (Amazon: USD 36.00), released in August 2019, comes with the powerful nRF52840 processor from Nordic Semiconductors, a 32-bit ARM Cortex-M4 CPU running at 64 MHz, 1MB of CPU Flash Memory, 256KB of SRAM, and a NINA-B306 stand-alone Bluetooth 5 low energy (BLE) module.

The Sense contains an impressive array of embedded sensors:
- 9-axis Inertial Sensor (LSM9DS1): 3D digital linear acceleration sensor, a 3D digital
angular rate sensor, and a 3D digital magnetic sensor - Humidity and Temperature Sensor (HTS221): Capacitive digital sensor for relative humidity and temperature
- Barometric Sensor (LPS22HB): MEMS nano pressure sensor: 260–1260 hectopascal (hPa) absolute digital output barometer
- Microphone (MP34DT05): MEMS audio sensor omnidirectional digital microphone
- Gesture, Proximity, Light Color, and Light Intensity Sensor (APDS9960): Advanced Gesture detection, Proximity detection, Digital Ambient Light Sense (ALS), and Color Sense (RGBC).
The Arduino Sense is an excellent, low-cost single-board microcontroller for learning about the collection and transmission of IoT sensor data.
IoT Gateway
An IoT Gateway, according to TechTarget, is a physical device or software program that serves as the connection point between the Cloud and controllers, sensors, and intelligent devices. All data moving to the Cloud, or vice versa, goes through the gateway, which can be either a dedicated hardware appliance or software program.
LoRa Gateways, to paraphrase The Things Network, form the bridge between devices and the Cloud. Devices use low power networks like LoRaWAN to connect to the Gateway, while the Gateway uses high bandwidth networks like WiFi, Ethernet, or Cellular to connect to the Cloud.

A third-generation Raspberry Pi 3 Model B+ single-board computer (SBC) will serve as our LoRa IoT Gateway. This Raspberry Pi model features a 1.4GHz Cortex-A53 (ARMv8) 64-bit quad-core processor System on a Chip (SoC), 1GB LPDDR2 SDRAM, dual-band wireless LAN, Bluetooth 4.2 BLE, and Gigabit Ethernet (Amazon: USD 42.99).

LoRa Transceiver Modules
To transmit the IoT sensor data between the IoT device, containing the embedded sensors, and the IoT gateway, I have used the REYAX RYLR896 LoRa transceiver module (Amazon: USD 19.50 x 2). The transceiver modules are commonly referred to as a universal asynchronous receiver-transmitter (UART). A UART is a computer hardware device for asynchronous serial communication in which the data format and transmission speeds are configurable.
According to the manufacturer, REYAX, the RYLR896 contains the Semtech SX1276 long-range, low power transceiver. The RYLR896 module provides ultra-long range spread spectrum communication and high interference immunity while minimizing current consumption. Each RYLR896 module contains a small, PCB integrated, helical antenna. This transceiver operates at both the 868 and 915 MHz frequency ranges. In this demonstration, we will be transmitting at 915 MHz for North America.
The Arduino Sense (IoT device) transmits data, using one of the RYLR896 modules (shown below front). The Raspberry Pi (IoT Gateway), connected to the other RYLR896 module (shown below rear), receives the data.

LoRaWAN Security
The RYLR896 is capable of AES 128-bit data encryption. Using the Advanced Encryption Standard (AES), we will encrypt the data sent from the IoT device to the IoT gateway, using a 32 hex digit password (128 bits / 4 bits/hex digit).
Provisioning AWS Resources
To start, we will create the necessary AWS IoT and associated resources on the AWS cloud platform. Once these resources are in place, we can then proceed to configure the IoT device and IoT gateway to securely transmit the sensor data to the Cloud.
All the source code for this post is on GitHub. Use the following command to git clone a local copy of the project.
git clone --branch master --single-branch --depth 1 --no-tags \
https://github.com/garystafford/iot-aws-lora-demo.git
AWS CloudFormation
The CloudFormation template, iot-analytics.yaml, will create an AWS IoT CloudFormation stack containing the following resources.
- AWS IoT Thing
- AWS IoT Thing Policy
- AWS IoT Core Topic Rule
- AWS IoT Analytics Channel, Pipeline, Data store, and Data set
- AWS Lambda and Lambda Permission
- Amazon S3 Bucket
- Amazon SageMaker Notebook Instance
- AWS IAM Roles
Please be aware of the costs involved with the AWS resources used in the CloudFormation template before continuing. To create the AWS CloudFormation stack from the included CloudFormation template, execute the following AWS CLI command.
The resulting CloudFormation stack should contain 16 AWS resources.

Additional Resources
Unfortunately, AWS CloudFormation cannot create all the AWS IoT resources we require for this demonstration. To complete the AWS provisioning process, execute the following series of AWS CLI commands, aws_cli_commands.md. These commands will create the remaining resources, including an AWS IoT Thing Type, Thing Group, Thing Billing Group, and an X.509 Certificate.
IoT Device Configuration
With the AWS resources deployed, we can configure the IoT device and IoT Gateway.
Arduino Sketch
For those not familiar with Arduino, a sketch is the name that Arduino uses for a program. It is the unit of code that is uploaded into non-volatile flash memory and runs on an Arduino board. The Arduino language is a set of C and C++ functions. All standard C and C++ constructs supported by the avr-g++ compiler should work in Arduino.
For this post, the sketch, lora_iot_demo_aws.ino, contains the code necessary to collect and securely transmit the environmental sensor data, including temperature, relative humidity, barometric pressure, Red, Green, and Blue (RGB) color, and ambient light intensity, using the LoRaWAN protocol.
AT Commands
Communications with the RYLR896’s long-range modem is done using AT commands. AT commands are instructions used to control a modem. AT is the abbreviation of ATtention. Every command line starts with AT. That is why modem commands are called AT commands, according to Developer’s Home. A complete list of AT commands can be downloaded as a PDF from the RYLR896 product page.
To efficiently transmit the environmental sensor data from the IoT sensor to the IoT gateway, the sketch concatenates the sensor ID and the sensor values together in a single string. The string will be incorporated into an AT command, sent to the RYLR896 LoRa transceiver module. To make it easier to parse the sensor data on the IoT gateway, we will delimit the sensor values with a pipe (|), as opposed to a comma. According to REYAX, the maximum length of the LoRa payload is approximately 330 bytes.
Below, we see an example of an AT command used to send the sensor data from the IoT sensor and the corresponding unencrypted data received by the IoT gateway. Both contain the LoRa transmitter Address ID, payload length (62 bytes in the example), and the payload. The data received by the IoT gateway also has the Received signal strength indicator (RSSI), and Signal-to-noise ratio (SNR).
Receiving Data on IoT Gateway
The Raspberry Pi will act as a LoRa IoT gateway, receiving the environmental sensor data from the IoT device, the Arduino, and sending the data to AWS. The Raspberry Pi runs a Python script, rasppi_lora_receiver_aws.py, which will receive the data from the Arduino Sense, decrypt the data, parse the sensor values, and serialize the data to a JSON payload, and finally, transmit the payload in an MQTT-protocol message to AWS. The script uses the pyserial, the Python Serial Port Extension, which encapsulates the access for the serial port for communication with the RYLR896 module. The script uses the AWS IoT Device SDK for Python v2 to communicate with AWS.
Running the IoT Gateway Python Script
To run the Python script on the Raspberry Pi, we will use a helper shell script, rasppi_lora_receiver_aws.sh. The shell script helps construct the arguments required to execute the Python script.
To run the helper script, we execute the following command, substituting the input parameter, the AWS IoT endpoint, with your endpoint.
sh ./rasppi_lora_receiver_aws.sh \
a1b2c3d4e5678f-ats.iot.us-east-1.amazonaws.com
You should see the console output, similar to the following.

The script starts by configuring the RYLR896 module and outputting that configuration to a log file, output.log
. If successful, we should see the following debug information logged.
DEBUG:root:Connecting to a1b2c3d4e5f6-ats.iot.us-east-1.amazonaws.com with client ID 'lora-iot-gateway-01'
DEBUG:root:Connecting to REYAX RYLR896 transceiver module
DEBUG:root:Connected!
DEBUG:root:Address set? +OK
DEBUG:root:Network Id set? +OK
DEBUG:root:AES-128 password set? +OK
DEBUG:root:Module responding? +OK
DEBUG:root:Address: +ADDRESS=116
DEBUG:root:Network id: +NETWORKID=6
DEBUG:root:UART baud rate: +IPR=115200
DEBUG:root:RF frequency: +BAND=915000000
DEBUG:root:RF output power: +CRFOP=15
DEBUG:root:Work mode: +MODE=0
DEBUG:root:RF parameters: +PARAMETER=12,7,1,4
DEBUG:root:AES128 password of the network: +CPIN=92A0ECEC9000DA0DCF0CAAB0ABA2E0EF
That sensor data is also written to the log file for debugging purposes. This first line in the log (shown below) is the raw decrypted data received from the IoT device via LoRaWAN. The second line is the JSON-serialized payload, sent securely to AWS, using the MQTT protocol.
DEBUG:root:b'+RCV=116,59,0447383033363932003C0034|23.46|41.89|99.38|230|692|833|1116,-48,39\r\n'
DEBUG:root:{'ts': 1598305503.7041512, 'data': {'humidity': 41.89, 'temperature': 23.46, 'device_id': '0447383033363932003C0034', 'gateway_id': '00000000f62051ce', 'pressure': 99.38, 'color': {'red': 230.0, 'blue': 833.0, 'ambient': 1116.0, 'green': 692.0}}}
DEBUG:root:b'+RCV=116,59,0447383033363932003C0034|23.46|41.63|99.38|236|696|837|1127,-49,35\r\n'
DEBUG:root:{'ts': 1598305513.7918658, 'data': {'humidity': 41.63, 'temperature': 23.46, 'device_id': '0447383033363932003C0034', 'gateway_id': '00000000f62051ce', 'pressure': 99.38, 'color': {'red': 236.0, 'blue': 837.0, 'ambient': 1127.0, 'green': 696.0}}}
DEBUG:root:b'+RCV=116,59,0447383033363932003C0034|23.44|41.57|99.38|232|686|830|1113,-48,32\r\n'
DEBUG:root:{'ts': 1598305523.8556132, 'data': {'humidity': 41.57, 'temperature': 23.44, 'device_id': '0447383033363932003C0034', 'gateway_id': '00000000f62051ce', 'pressure': 99.38, 'color': {'red': 232.0, 'blue': 830.0, 'ambient': 1113.0, 'green': 686.0}}}
DEBUG:root:b'+RCV=116,59,0447383033363932003C0034|23.51|41.44|99.38|205|658|802|1040,-48,36\r\n'
DEBUG:root:{'ts': 1598305528.8890748, 'data': {'humidity': 41.44, 'temperature': 23.51, 'device_id': '0447383033363932003C0034', 'gateway_id': '00000000f62051ce', 'pressure': 99.38, 'color': {'red': 205.0, 'blue': 802.0, 'ambient': 1040.0, 'green': 658.0}}}
AWS IoT Core
The Raspberry Pi-based IoT gateway will be registered with AWS IoT Core. IoT Core allows users to connect devices quickly and securely to AWS.
Things
According to AWS, IoT Core can reliably scale to billions of devices and trillions of messages. Registered devices are referred to as things in AWS IoT Core. A thing is a representation of a specific device or logical entity. Information about a thing is stored in the registry as JSON data.
Below, we see an example of the Thing created by CloudFormation. The Thing, lora-iot-gateway-01
, represents the physical IoT gateway. We have assigned the IoT gateway a Thing Type, LoRaIoTGateway
, a Thing Group, LoRaIoTGateways
, and a Thing Billing Group, IoTGateways
.

In a real IoT environment, containing hundreds, thousands, even millions of IoT devices, gateways, and sensors, these classification mechanisms, Thing Type, Thing Group, and Thing Billing Group, will help to organize IoT assets.

Device Gateway and Message Broker
IoT Core provides a Device Gateway, which manages all active device connections. The Gateway currently supports MQTT, WebSockets, and HTTP 1.1 protocols. Behind the Message Gateway is a high-throughput pub/sub Message Broker, which securely transmits messages to and from all IoT devices and applications with low latency. Below, we see a typical AWS IoT Core architecture containing multiple Topics, Rules, and Actions.

AWS IoT Security
AWS IoT Core provides mutual authentication and encryption, ensuring all data is exchanged between AWS and the devices are secure by default. In the demonstration, all data is sent securely using Transport Layer Security (TLS) 1.2 with X.509 digital certificates on port 443. Below, we see an example of an X.509 certificate assigned to the Thing, lora-iot-gateway-01
, which represents the physical IoT gateway. The X.509 certificate and the private key, generated using the AWS CLI, previously, are installed on the IoT gateway.

Authorization of the device to access any resource on AWS is controlled by AWS IoT Core Policies. These policies are similar to AWS IAM Policies. Below, we see an example of an AWS IoT Core Policy, LoRaDevicePolicy
, which is assigned to the IoT gateway.

AWS IoT Core Rules
Once an MQTT message is received from the IoT gateway (a thing), we use AWS IoT Rules to send message data to an AWS IoT Analytics Channel. Rules give your devices the ability to interact with AWS services. Rules are analyzed, and Actions are performed based on the MQTT topic stream. Below, we see an example rule that forwards our messages to an IoT Analytics Channel.

Rule query statements are written in standard Structured Query Language (SQL). The datasource for the Rule query is an IoT Topic.
AWS IoT Analytics
AWS IoT Analytics is composed of five primary components: Channels, Pipelines, Data stores, Data sets, and Notebooks. These components enable you to collect, prepare, store, analyze, and visualize your IoT data.

Below, we see a typical AWS IoT Analytics architecture. IoT messages are received from AWS IoT Core, thought a Rule Action. Amazon QuickSight provides business intelligence, visualization. Amazon QuickSight ML Insights adds anomaly detection and forecasting.

IoT Analytics Channel
An AWS IoT Analytics Channel pulls messages or data into IoT Analytics from other AWS sources, such as Amazon S3, Amazon Kinesis, or Amazon IoT Core. Channels store data for IoT Analytics Pipelines. Both Channels and Data store support storing data in your own Amazon S3 bucket or an IoT Analytics service-managed S3 bucket. In the demonstration, we are using a service managed S3 bucket.
When creating a Channel, you also decide how long to retain the data. For the demonstration, we have set the data retention period for 21 days. Channels are generally not used for long term storage of data. Typically, you would only retain data in the Channel for the period you need to analyze. For long term storage of IoT message data, I recommend using an AWS IoT Core Rule to send a copy of the raw IoT data to Amazon S3, using a service such as Amazon Kinesis Data Firehose.

IoT Analytics Pipeline
An AWS IoT Analytics Pipeline consumes messages from one or more Channels. Pipelines transform, filter, and enrich the messages before storing them in IoT Analytics Data stores. A Pipeline is composed of an ordered list of activities. Logically, you must specify both a Channel
(source) and a Datastore
(destination) activity. Optionally, you may choose as many as 23 additional activities in the pipelineActivities
array.
In our demonstration’s Pipeline, iot_analytics_pipeline
, we have specified three additional activities, including DeviceRegistryEnrich
, Filter
, and Lambda
. Other activity types include Math
, SelectAttributes
, RemoveAttributes,
and AddAttributes
.

The Filter
activity ensures the sensor values are not Null or otherwise erroneous; if true, the message is dropped. The Lambda
Pipeline activity executes an AWS Lambda function to transform the messages in the pipeline. Messages are sent in an event object to the Lambda. The message is modified, and the event object is returned to the activity.

The Python-based Lambda function easily handles typical IoT data transformation tasks, including converting the temperature
from Celsius to Fahrenheit, pressure
from kilopascals (kPa) to inches of Mercury (inHg), and 12-bit RGBA values to 8-bit color values (0–255). The Lambda function also rounds down all the values to between 0 and 2 decimal places of precision.
The demonstration’s Pipeline also enriches the IoT data with metadata from the IoT device’s AWS IoT Core Registry. The metadata includes additional information about the device that generated the IoT data, including the custom attributes such as LoRa transceiver manufacturer and model, and the IoT gateway manufacturer.

A notable feature of Pipelines is the ability to reprocess messages. If you make changes to the Pipeline, which often happens during the data preparation stage, you can reprocess any or all the IoT data in the associated Channel, and overwrite the IoT data in the Data set.
IoT Analytics Data store
An AWS IoT Analytics Data store stores prepared data from an AWS IoT Analytics Pipeline, in a fully-managed database. Both Channels and Data store support storing IoT data in your own Amazon S3 bucket or an IoT Analytics managed S3 bucket. In the demonstration, we are using a service-managed S3 bucket to store the IoT data in our Data store, iot_analytics_data_store
.

IoT Analytics Data set
An AWS IoT Analytics Data set automatically provides regular, up-to-date insights for data analysts by querying a Data store using standard SQL. Periodic updates are implemented using a cron expression. For the demonstration, we are updating our Data set, iot_analytics_data_set
, at a 15-minute interval. The time interval can be increased or reduced, depending on the desired ‘near real-time’ nature of the IoT data being analyzed.
Below, we see messages in the Result preview pane of the Data set. Note the SQL query used to obtain the messages, which queries the Data store. The Data store, as you will recall, contains the transformed messages from the Pipeline.

IoT Analytics Data sets also support sending content results, which are materialized views of your IoT Analytics data, to an Amazon S3 bucket.

The CloudFormation stack created an encrypted Amazon S3 Bucket. This bucket receives a copy of the messages from the IoT Analytics Data set whenever the cron expression runs the scheduled update.

IoT Analytics Notebook
An AWS IoT Analytics Notebook allows users to perform statistical analysis and machine learning on IoT Analytics Data sets using Jupyter Notebooks. The IoT Analytics Notebook service includes a set of notebook templates that contain AWS-authored machine learning models and visualizations. Notebook Instances can be linked to a GitHub or other source code repository. Notebooks created with IoT Analytics Notebook can also be accessed directly through Amazon SageMaker. For the demonstration, the Notebooks Instance is cloned from our project’s GitHub repository.

The repository contains a sample Jupyter Notebook, LoRa_IoT_Analytics_Demo.ipynb, based on the conda_python3
kernel. This preinstalled environment includes the default Anaconda installation and Python 3.

The Notebook uses pandas, matplotlib, and plotly to manipulate and visualize the sample IoT data stored in the IoT Analytics Data set.



The Notebook can be modified, and the changes pushed back to GitHub. You could easily fork the demonstration’s GitHub repository and modify the CloudFormation template to point to your source code repository.

Amazon QuickSight
Amazon QuickSight provides business intelligence (BI) and visualization. Amazon QuickSight ML Insights adds anomaly detection and forecasting. We can use Amazon QuickSight to visualize the IoT message data, stored in the IoT Analytics Data set.
Amazon QuickSight has both a Standard and an Enterprise Edition. AWS provides a detailed product comparison of each edition. For the post, I am demonstrating the Enterprise Edition, which includes additional features, such as ML Insights, hourly refreshes of SPICE (super-fast, parallel, in-memory, calculation engine), and theme customization.
Please be aware of the costs of Amazon QuickSight if you choose to follow along with this part of the demo. Although there is an Amazon QuickSight API, Amazon QuickSight is not automatically enabled or configured with CloudFormation or using the AWS CLI in this demonstration.
QuickSight Data Sets
Amazon QuickSight has a wide variety of data source options for creating Amazon QuickSight Data sets, including the ones shown below. Do not confuse Amazon QuickSight Data sets with IoT Analytics Data sets; they are two different service features.

For the demonstration, we will create an Amazon QuickSight Data set that will use our IoT Analytics Data set, iot_analytics_data_set
.

Amazon QuickSight gives you the ability to view and modify QuickSight Data sets before visualizing. QuickSight even provides a wide variety of functions, enabling us to perform dynamic calculations on the field values. For this demonstration, we will leave the data unchanged since all transformations were already completed in the IoT Analytics Pipeline.

QuickSight Analysis
Using the QuickSight Data set, built from the IoT Analytics Data set as a data source, we create a QuickSight Analysis. The QuickSight Analysis console is shown below. An Analysis is primarily a collection of Visuals (aka Visual types). QuickSight provides several Visual types. Each visual is associated with a Data set. Data for the QuickSight Analysis or each visual within the Analysis can be filtered. For the demo, I have created a simple QuickSight Analysis, including a few typical QuickSight visuals.

QuickSight Dashboards
To share a QuickSight Analysis, we can create a QuickSight Dashboard. Below, we see a few views of the QuickSight Analysis, shown above, as a Dashboard. Although viewers of the Dashboard cannot edit the visuals, they can apply filtering and interactively drill-down into data in the Visuals.



Amazon QuickSight ML Insights
According to Amazon, ML Insights leverages AWS’s machine learning (ML) and natural language capabilities to gain deeper insights from data. QuickSight’s ML-powered Anomaly Detection continuously analyze data to discover anomalies and variations inside of the aggregates, giving you the insights to act when business changes occur. QuickSight’s ML-powered Forecasting can be used to predict your business metrics accurately, and perform interactive what-if analysis with point-and-click simplicity. QuickSight’s built-in algorithms make it easy for anyone to use ML that learns from your data patterns to provide you with accurate predictions based on historical trends.
Below, we see the ML Insights tab (left) in the demonstration’s QuickSight Analysis. Individually detected anomalies can be added to the QuickSight Analysis, like Visuals, and configured to tune the detection parameters. Observe the temperature, humidity, and barometric pressure anomalies, identified by ML Insights, based on their Anomaly Score, which is higher or lower, given a minimum delta of five percent. These anomalies accurately reflected an actual failure of the IoT device, caused by overheated during testing, which resulted in abnormal sensor readings.

Receiving the Messages on AWS
To confirm the IoT gateway is sending messages, we can use a packet analyzer, like tcpdump
, on the IoT gateway. Running tcpdump
on the IoT gateway, below, we see outbound encrypted MQTT messages being sent to AWS on port 443.

To confirm those messages are being received from the IoT gateway on AWS, we can use the AWS IoT Core Test feature and subscribe to the lora-iot-demo
topic. We should see messages flowing in from the IoT gateway at approximately 5-second intervals.

The JSON payload structure of the incoming MQTT messages will look similar to the below example. The device_id
is the unique id of the IoT device that transmitted the message using LoRaWAN. The gateway_id
is the unique id of the IoT gateway that received the message using LoRaWAN and sent it to AWS. A single IoT gateway would usually manage messages from multiple IoT devices, each with a unique id.
The SQL query used by the AWS IoT Rule described earlier, transforms and flattens the nested JSON payload structure, before passing it to the AWS IoT Analytics Channel, as shown below.
We can measure the near real-time nature of the IoT data using the ts
and msg_received
data fields. The ts
data field is date and time when the sensor reading occurred on the IoT device, while the msg_received
data field is the date and time when the message was received on AWS. The delta between the two values is a measure of how near real-time the sensor readings are being streamed to the AWS IoT Analytics Channel. In the below example, the difference between ts
(2020–08–27T11:45:31.986) and msg_received
(2020–08–27T11:45:32.074) is 88 ms.
Final IoT Data Message Structure
Once the message payload passes through the AWS IoT Analytics Pipeline and lands in the AWS IoT Analytics Data set, its final data structure looks as follows. Note that the device’s attribute metadata has been added from the AWS IoT Core device registry. Regrettably, the metadata is not well-formatted JSON and will require additional transformation to be usable.
A set of sample messages is included in the GitHub project’s sample_messages directory.
Conclusion
In this post, we explored the use of the LoRa and LoRaWAN protocols to transmit environmental sensor data from an IoT device to an IoT gateway. Given its low energy consumption, long-distance transmission capabilities, and well-developed protocols, LoRaWAN is an ideal long-range wireless protocol for IoT devices. We then demonstrated how to use AWS IoT Device SDKs, AWS IoT Core, and AWS IoT Analytics to securely collect, analyze, and visualize streaming messages from the IoT device, in near real-time.
This blog represents my own viewpoints and not of my employer, Amazon Web Services.
LoRa and LoRaWAN for IoT: Getting Started with LoRa and LoRaWAN Protocols for Low Power, Wide Area Networking of IoT
Posted by Gary A. Stafford in C++ Development, IoT, Python, Raspberry Pi, Software Development, Technology Consulting on August 10, 2020
Introduction
According to the LoRa Alliance, Low-Power, Wide-Area Networks (LPWAN) are projected to support a major portion of the billions of devices forecasted for the Internet of Things (IoT). LoRaWAN is designed from the bottom up to optimize LPWANs for battery lifetime, capacity, range, and cost. LoRa and LoRaWAN permit long-range connectivity for the Internet of Things (IoT) devices in different types of industries. According to Wikipedia, LoRaWAN defines the communication protocol and system architecture for the network, while the LoRa physical layer enables the long-range communication link.
LoRa
Long Range (LoRa), the low-power wide-area network (LPWAN) protocol developed by Semtech, sits at layer 1, the physical layer, of the seven-layer OSI model (Open Systems Interconnection model) of computer networking. The physical layer defines the means of transmitting raw bits over a physical data link connecting network nodes. LoRa uses license-free sub-gigahertz radio frequency (RF) bands, including 433 MHz, 868 MHz (Europe), 915 MHz (Australia and North America), and 923 MHz (Asia). LoRa enables long-range transmissions with low power consumption.
LoRaWAN
LoRaWAN is a cloud-based medium access control (MAC) sublayer (layer 2) protocol but acts mainly as a network layer (layer 3) protocol for managing communication between LPWAN gateways and end-node devices as a routing protocol, maintained by the LoRa Alliance. The MAC sublayer and the logical link control (LLC) sublayer together make up layer 2, the data link layer, of the OSI model.
LoRaWAN is often cited as having greater than a 10-km-wide coverage area in rural locations. However, according to other sources, it is generally more limited. According to the Electronic Design article, 11 Myths About LoRaWAN, a typical LoRaWAN network range depends on numerous factors—indoor or outdoor gateways, the payload of the message, the antenna used, etc. On average, in an urban environment with an outdoor gateway, you can expect up to 2- to 3-km-wide coverage, while in the rural areas it can reach beyond 5 to 7 km. LoRa’s range depends on the “radio line-of-sight.” Radio waves in the 400 MHz to 900 MHz range may pass through some obstructions, depending on their composition, but will be absorbed or reflected otherwise. This means that the signal can potentially reach as far as the horizon, as long as there are no physical barriers to block it.
In the following hands-on post, we will explore the use of the LoRa and LoRaWAN protocols to transmit and receive sensor data, over a substantial distance, between an IoT device, containing a number of embedded sensors, and an IoT gateway.
Recommended Hardware
For this post, I have used the following hardware.
IoT Device with Embedded Sensors
I have used an Arduino single-board microcontroller as an IoT sensor, actually an array of sensors. The 3.3V AI-enabled Arduino Nano 33 BLE Sense board (Amazon: USD 36.00), released in August 2019, comes with the powerful nRF52840 processor from Nordic Semiconductors, a 32-bit ARM Cortex-M4 CPU running at 64 MHz, 1MB of CPU Flash Memory, 256KB of SRAM, and a NINA-B306 stand-alone Bluetooth 5 low energy (BLE) module.
The Sense also contains an impressive array of embedded sensors:
- 9-axis Inertial Sensor (LSM9DS1): 3D digital linear acceleration sensor, a 3D digital
angular rate sensor, and a 3D digital magnetic sensor - Humidity and Temperature Sensor (HTS221): Capacitive digital sensor for relative humidity and temperature
- Barometric Sensor (LPS22HB): MEMS nano pressure sensor: 260–1260 hectopascal (hPa) absolute digital output barometer
- Microphone (MP34DT05): MEMS audio sensor omnidirectional digital microphone
- Gesture, Proximity, Light Color, and Light Intensity Sensor (APDS9960): Advanced Gesture detection, Proximity detection, Digital Ambient Light Sense (ALS), and Color Sense (RGBC).
The Arduino Sense is an excellent, low-cost single-board microcontroller for learning about the collection and transmission of IoT sensor data.
IoT Gateway
An IoT Gateway, according to TechTarget, is a physical device or software program that serves as the connection point between the Cloud and controllers, sensors, and intelligent devices. All data moving to the Cloud, or vice versa goes through the gateway, which can be either a dedicated hardware appliance or software program.
I have used an a third-generation Raspberry Pi 3 Model B+ single-board computer (SBC), to serve as an IoT Gateway. This Raspberry Pi model features a 1.4GHz Cortex-A53 (ARMv8) 64-bit quad-core processor System on a Chip (SoC), 1GB LPDDR2 SDRAM, dual-band wireless LAN, Bluetooth 4.2 BLE, and Gigabit Ethernet (Amazon: USD 42.99).
To follow along with the post, you could substitute the Raspberry Pi for any Linux-based machine to run the included sample Python script.
LoRa Transceiver Modules
To transmit the IoT sensor data between the IoT device, containing the embedded sensors, and the IoT gateway, I have used the REYAX RYLR896 LoRa transceiver module (Amazon: USD 19.50 x 2). The transceiver modules are commonly referred to as a universal asynchronous receiver-transmitter (UART). A UART is a computer hardware device for asynchronous serial communication in which the data format and transmission speeds are configurable.
According to the manufacturer, REYAX, the RYLR896 contains the Semtech SX1276 long-range, low power transceiver. The RYLR896 module provides ultra-long range spread spectrum communication and high interference immunity while minimizing current consumption. This transceiver operates at both the 868 and 915 MHz frequency ranges. We will be transmitting at 915 MHz for North America, in this post. Each RYLR896 module contains a small, PCB integrated, helical antenna.
Security
The RYLR896 is capable of the AES 128-bit data encryption. Using the Advanced Encryption Standard (AES), we will encrypt the data sent from the IoT device to the IoT gateway, using a 32 hex digit password (128 bits / 4 bits/hex digit = 32 hex digits). Using hexadecimal notation, the password is limited to digits 0–9 and characters A–F.
USB to TTL Serial Converter Adapter
Optionally, to configure, test, and debug the RYLR896 LoRa transceiver module directly from your laptop, you can use a USB to TTL serial converter adapter. I currently use the IZOKEE FT232RL FTDI USB to TTL Serial Converter Adapter Module for 3.3V and 5V (Amazon: USD 9.49 for 2). The 3.3V RYLR896 module easily connects to the USB to TTL Serial Converter Adapter using the TXD/TX, RXD/RX, VDD/VCC, and GND pins. We use serial communication to send and receive data through TX (transmit) and RX (receive) pins. The wiring is shown below: VDD to VCC, GND to GND, TXD to RX, and RXD to TX.
The FT232RL has support for baud rates up to 115,200 bps, which is the speed we will use to communicate with the RYLR896 module.
Arduino Sketch
For those not familiar with Arduino, a sketch is the name that Arduino uses for a program. It is the unit of code that is uploaded into non-volatile flash memory and runs on an Arduino board. The Arduino language is a set of C and C++ functions. All standard C and C++ constructs supported by the avr-g++ compiler should work in Arduino.
For this post, the sketch, lora_iot_demo.ino, contains all the code necessary to collect and securely transmit the environmental sensor data, including temperature, relative humidity, barometric pressure, RGB color, and ambient light intensity, using the LoRaWAN protocol. All code for this post, including the sketch, can be found on GitHub.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
Description: Transmits Arduino Nano 33 BLE Sense sensor telemetry over LoRaWAN, | |
including temperature, humidity, barometric pressure, and color, | |
using REYAX RYLR896 transceiver modules | |
http://reyax.com/wp-content/uploads/2020/01/Lora-AT-Command-RYLR40x_RYLR89x_EN.pdf | |
Author: Gary Stafford | |
*/ | |
#include <Arduino_HTS221.h> | |
#include <Arduino_LPS22HB.h> | |
#include <Arduino_APDS9960.h> | |
const int UPDATE_FREQUENCY = 5000; // update frequency in ms | |
const float CALIBRATION_FACTOR = –4.0; // temperature calibration factor (Celsius) | |
const int ADDRESS = 116; | |
const int NETWORK_ID = 6; | |
const String PASSWORD = "92A0ECEC9000DA0DCF0CAAB0ABA2E0EF"; | |
const String DELIMITER = "|"; | |
void setup() | |
{ | |
Serial.begin(9600); | |
Serial1.begin(115200); // default baud rate of module is 115200 | |
delay(1000); // wait for LoRa module to be ready | |
// needs all need to be same for receiver and transmitter | |
Serial1.print((String)"AT+ADDRESS=" + ADDRESS + "\r\n"); | |
delay(200); | |
Serial1.print((String)"AT+NETWORKID=" + NETWORK_ID + "\r\n"); | |
delay(200); | |
Serial1.print("AT+CPIN=" + PASSWORD + "\r\n"); | |
delay(200); | |
Serial1.print("AT+CPIN?\r\n"); // confirm password is set | |
if (!HTS.begin()) | |
{ // initialize HTS221 sensor | |
Serial.println("Failed to initialize humidity temperature sensor!"); | |
while (1); | |
} | |
if (!BARO.begin()) | |
{ // initialize LPS22HB sensor | |
Serial.println("Failed to initialize pressure sensor!"); | |
while (1); | |
} | |
// avoid bad readings to start bug | |
// https://forum.arduino.cc/index.php?topic=660360.0 | |
BARO.readPressure(); | |
delay(1000); | |
if (!APDS.begin()) | |
{ // initialize APDS9960 sensor | |
Serial.println("Failed to initialize color sensor!"); | |
while (1); | |
} | |
} | |
void loop() | |
{ | |
updateReadings(); | |
delay(UPDATE_FREQUENCY); | |
} | |
void updateReadings() | |
{ | |
float temperature = getTemperature(CALIBRATION_FACTOR); | |
float humidity = getHumidity(); | |
float pressure = getPressure(); | |
int colors[4]; | |
getColor(colors); | |
String payload = buildPayload(temperature, humidity, pressure, colors); | |
// Serial.println("Payload: " + payload); // display the payload for debugging | |
Serial1.print(payload); // send the payload over LoRaWAN WiFi | |
displayResults(temperature, humidity, pressure, colors); // display the results for debugging | |
} | |
float getTemperature(float calibration) | |
{ | |
return HTS.readTemperature() + calibration; | |
} | |
float getHumidity() | |
{ | |
return HTS.readHumidity(); | |
} | |
float getPressure() | |
{ | |
return BARO.readPressure(); | |
} | |
void getColor(int c[]) | |
{ | |
// check if a color reading is available | |
while (!APDS.colorAvailable()) | |
{ | |
delay(5); | |
} | |
int r, g, b, a; | |
APDS.readColor(r, g, b, a); | |
c[0] = r; | |
c[1] = g; | |
c[2] = b; | |
c[3] = a; | |
} | |
void displayResults(float t, float h, float p, int c[]) | |
{ | |
Serial.print("Temperature: "); | |
Serial.println(t); | |
Serial.print("Humidity: "); | |
Serial.println(h); | |
Serial.print("Pressure: "); | |
Serial.println(p); | |
Serial.print("Color (r, g, b, a): "); | |
Serial.print(c[0]); | |
Serial.print(", "); | |
Serial.print(c[1]); | |
Serial.print(", "); | |
Serial.print(c[2]); | |
Serial.print(", "); | |
Serial.println(c[3]); | |
Serial.println("———-"); | |
} | |
String buildPayload(float t, float h, float p, int c[]) | |
{ | |
String readings = ""; | |
readings += t; | |
readings += DELIMITER; | |
readings += h; | |
readings += DELIMITER; | |
readings += p; | |
readings += DELIMITER; | |
readings += c[0]; | |
readings += DELIMITER; | |
readings += c[1]; | |
readings += DELIMITER; | |
readings += c[2]; | |
readings += DELIMITER; | |
readings += c[3]; | |
String payload = ""; | |
payload += "AT+SEND="; | |
payload += ADDRESS; | |
payload += ","; | |
payload += readings.length(); | |
payload += ","; | |
payload += readings; | |
payload += "\r\n"; | |
return payload; | |
} |
AT Commands
Communications with the RYLR896’s long-range modem is done using AT commands. AT commands are instructions used to control a modem. AT is the abbreviation of ATtention. Every command line starts with “AT”. That is why modem commands are called AT commands, according to Developer’s Home. A complete list of AT commands can be downloaded as a PDF from the RYLR896 product page.
To efficiently transmit the environmental sensor data from the IoT sensor to the IoT gateway, the sketch concatenates the sensor values together in a single string. The string will be incorporated into AT command to send the data to the RYLR896 LoRa transceiver module. To make it easier to parse the sensor data on the IoT gateway, we will delimit the sensor values with a pipe (|), as opposed to a comma. The maximum length of the payload (sensor data) is 240 bytes.
Below, we see an example of an AT command used to send the sensor data from the IoT sensor and the corresponding unencrypted data received by the IoT gateway. Both strings contain the LoRa transmitter Address ID, payload length, and the payload. The data received by the IoT gateway also contains the Received signal strength indicator (RSSI), and Signal-to-noise ratio (SNR).
Configure, Test, and Debug
As discussed earlier, to configure, test, and debug the RYLR896 LoRa transceiver modules without the use of the IoT gateway, you can use a USB to TTL serial converter adapter. The sketch is loaded on the Arduino Sense (the IoT device) and actively transmits data through one of the RYLR896 modules (shown below right). The other RYLR896 module is connected to your laptop’s USB port, via the USB to TTL serial converter adapter (shown below left). Using a terminal and the screen command, or the Arduino desktop application’s Serial Terminal, we can receive the sensor data from the Arduino Sense.
Using a terminal on your laptop, we first need to locate the correct virtual console (aka virtual terminal). On Linux or Mac, the virtual consoles are represented by device special files, such as /dev/tty1
, /dev/tty2
, and so forth. To find the virtual console for the USB to TTL serial converter adapter plugged into the laptop, use the following command.
ls -alh /dev/tty.*
We should see a virtual console with a name similar to /dev/tty.usbserial-
.
... /dev/tty.Bluetooth-Incoming-Port ... /dev/tty.GarysBoseQC35II-SPPDev ... /dev/tty.a483e767cbac-Bluetooth- ... /dev/tty.usbserial-A50285BI
To connect to the RYLR896 module via the USB to TTL serial converter adapter, using the virtual terminal, we use the screen
command and connect at a baud rate of 115,200 bps.
screen /dev/tty.usbserial-A50285BI 115200
If everything is configured and working correctly, we should see data being transmitted from the Arduino Sense and received by the local machine, at five second intervals. Each line of unencrypted data transmitted will look similar to the following, +RCV=116,25,22.18|41.57|99.74|2343|1190|543|4011,-34,47
. In the example below, the AES 128-bit data encryption is not enabled on the Arduino, yet. With encryption turned on the sensor data (the payload) would appear garbled.
Even easier than the screen
command, we can also use the Arduino desktop application’s Serial Terminal, as shown in the following short screen recording. Select the correct Port (virtual console) from the Tools menu and open the Serial Terminal. Since the transmitted data should be secured using AES 128-bit data encryption, we need to send an AT command (AT+CPIN
) containing the transceiver module’s common password, to correctly decrypt the data on the receiving device (e.g., AT+CPIN=92A0ECEC9000DA0DCF0CAAB0ABA2E0EF
).
Receiving Data on IoT Gateway
The Raspberry Pi will act as an IoT gateway, receiving the environmental sensor data from the IoT device, the Arduino. The Raspberry Pi will run a Python script, rasppi_lora_receiver.py, which will receive and decrypt the data payload, parse the sensor values, and display the values in the terminal. The script uses the pyserial, the Python Serial Port Extension. This Python module encapsulates the access for the serial port.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import logging | |
import time | |
from argparse import ArgumentParser | |
from datetime import datetime | |
import serial | |
from colr import color as colr | |
# LoRaWAN IoT Sensor Demo | |
# Using REYAX RYLR896 transceiver modules | |
# Author: Gary Stafford | |
# Requirements: python3 -m pip install –user -r requirements.txt | |
# To Run: python3 ./rasppi_lora_receiver.py –tty /dev/ttyAMA0 –baud-rate 115200 | |
# constants | |
ADDRESS = 116 | |
NETWORK_ID = 6 | |
PASSWORD = "92A0ECEC9000DA0DCF0CAAB0ABA2E0EF" | |
def main(): | |
logging.basicConfig(filename='output.log', filemode='w', level=logging.DEBUG) | |
args = get_args() # get args | |
payload = "" | |
print("Connecting to REYAX RYLR896 transceiver module…") | |
serial_conn = serial.Serial( | |
port=args.tty, | |
baudrate=int(args.baud_rate), | |
timeout=5, | |
parity=serial.PARITY_NONE, | |
stopbits=serial.STOPBITS_ONE, | |
bytesize=serial.EIGHTBITS | |
) | |
if serial_conn.isOpen(): | |
set_lora_config(serial_conn) | |
check_lora_config(serial_conn) | |
while True: | |
serial_payload = serial_conn.readline() # read data from serial port | |
if len(serial_payload) > 0: | |
try: | |
payload = serial_payload.decode(encoding="utf-8") | |
except UnicodeDecodeError: # receiving corrupt data? | |
logging.error("UnicodeDecodeError: {}".format(serial_payload)) | |
payload = payload[:–2] | |
try: | |
data = parse_payload(payload) | |
print("\n———-") | |
print("Timestamp: {}".format(datetime.now())) | |
print("Payload: {}".format(payload)) | |
print("Sensor Data: {}".format(data)) | |
display_temperature(data[0]) | |
display_humidity(data[1]) | |
display_pressure(data[2]) | |
display_color(data[3], data[4], data[5], data[6]) | |
except IndexError: | |
logging.error("IndexError: {}".format(payload)) | |
except ValueError: | |
logging.error("ValueError: {}".format(payload)) | |
# time.sleep(2) # transmission frequency set on IoT device | |
def eight_bit_color(value): | |
return int(round(value / (4097 / 255), 0)) | |
def celsius_to_fahrenheit(value): | |
return (value * 1.8) + 32 | |
def display_color(r, g, b, a): | |
print("12-bit Color values (r,g,b,a): {},{},{},{}".format(r, g, b, a)) | |
r = eight_bit_color(r) | |
g = eight_bit_color(g) | |
b = eight_bit_color(b) | |
a = eight_bit_color(a) # ambient light intensity | |
print(" 8-bit Color values (r,g,b,a): {},{},{},{}".format(r, g, b, a)) | |
print("RGB Color") | |
print(colr("\t\t", fore=(127, 127, 127), back=(r, g, b))) | |
print("Light Intensity") | |
print(colr("\t\t", fore=(127, 127, 127), back=(a, a, a))) | |
def display_pressure(value): | |
print("Barometric Pressure: {} kPa".format(round(value, 2))) | |
def display_humidity(value): | |
print("Humidity: {}%".format(round(value, 2))) | |
def display_temperature(value): | |
temperature = celsius_to_fahrenheit(value) | |
print("Temperature: {}°F".format(round(temperature, 2))) | |
def get_args(): | |
arg_parser = ArgumentParser(description="BLE IoT Sensor Demo") | |
arg_parser.add_argument("–tty", required=True, help="serial tty", default="/dev/ttyAMA0") | |
arg_parser.add_argument("–baud-rate", required=True, help="serial baud rate", default=1152000) | |
args = arg_parser.parse_args() | |
return args | |
def parse_payload(payload): | |
# input: +RCV=116,29,23.94|37.71|99.89|16|38|53|80,-61,56 | |
# output: [23.94, 37.71, 99.89, 16.0, 38.0, 53.0, 80.0] | |
payload = payload.split(",") | |
payload = payload[2].split("|") | |
payload = [float(i) for i in payload] | |
return payload | |
def set_lora_config(serial_conn): | |
# configures the REYAX RYLR896 transceiver module | |
serial_conn.write(str.encode("AT+ADDRESS=" + str(ADDRESS) + "\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("Address set?", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+NETWORKID=" + str(NETWORK_ID) + "\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("Network Id set?", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+CPIN=" + PASSWORD + "\r\n")) | |
time.sleep(1) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("AES-128 password set?", serial_payload.decode(encoding="utf-8")) | |
def check_lora_config(serial_conn): | |
serial_conn.write(str.encode("AT?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("Module responding?", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+ADDRESS?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("Address:", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+NETWORKID?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("Network id:", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+IPR?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("UART baud rate:", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+BAND?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("RF frequency", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+CRFOP?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("RF output power", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+MODE?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("Work mode", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+PARAMETER?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("RF parameters", serial_payload.decode(encoding="utf-8")) | |
serial_conn.write(str.encode("AT+CPIN?\r\n")) | |
serial_payload = (serial_conn.readline())[:–2] | |
print("AES128 password of the network", | |
serial_payload.decode(encoding="utf-8")) | |
if __name__ == "__main__": | |
main() |
Prior to running the Python script, we can test and debug the connection from the Arduino Sense to the Raspberry Pi using a general application such as Minicom. Minicom is a text-based modem control and terminal emulator program. To install Minicom on the Raspberry Pi, use the following command.
sudo apt-get install minicom
To run Minicom or the Python script, we will need to know the virtual console of the serial connection (Serial1
in the script) used to communicate with the RYLR896 module, wired to the Raspberry Pi. This can found using the following command.
dmesg | grep -E --color 'serial|tty'
Search for a line, similar to the last line, shown below. Note the name of the virtual console, in my case, ttyAMA0
.
[ 0.000000] Kernel command line: coherent_pool=1M bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x1ec00000 vc_mem.mem_size=0x20000000 dwc_otg.lpm_enable=0 console=tty1 root=PARTUUID=509d1565-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles [ 0.000637] console [tty1] enabled [ 0.863147] uart-pl011 20201000.serial: cts_event_workaround enabled [ 0.863289] 20201000.serial: ttyAMA0 at MMIO 0x20201000 (irq = 81, base_baud = 0) is a PL011 rev2
To view the data received from the Arduino Sense, using Minicom, use the following command, substituting the virtual console value, found above.
minicom -b 115200 -o -D /dev/ttyAMA0
If successful, we should see output similar to the lower right terminal window. Data is being transmitted by the Arduino Sense and being received by the Raspberry Pi, via LoRaWAN. In the below example, the AES 128-bit data encryption is not enabled on the Arduino, yet. With encryption turned on the sensor data (the payload) would appear garbled.
IoT Gateway Python Script
To run the Python script on the Raspberry Pi, use the following command, substituting the name of the virtual console (e.g., /dev/ttyAMA0
).
python3 ./rasppi_lora_receiver.py \ --tty /dev/ttyAMA0 --baud-rate 115200
The script starts by configuring the RYLR896 and outputting that configuration to the terminal. If successful, we should see the following informational output.
Connecting to REYAX RYLR896 transceiver module... Address set? +OK Network Id set? +OK AES-128 password set? +OK Module responding? +OK Address: +ADDRESS=116 Firmware version: +VER=RYLR89C_V1.2.7 Network Id: +NETWORKID=6 UART baud rate: +IPR=115200 RF frequency +BAND=915000000 RF output power +CRFOP=15 Work mode +MODE=0 RF parameters +PARAMETER=12,7,1,4 AES-128 password of the network +CPIN=92A0ECEC9000DA0DCF0CAAB0ABA2E0EF
Once configured, the script will receive the data from the Arduino Sense, decrypt the data, parse the sensor values, and format and display the values within the terminal.
The following screen recording shows a parallel view of both the Arduino Serial Monitor (upper right window) and the Raspberry Pi’s terminal output (lower right window). The Raspberry Pi (receiver) receives data from the Arduino (transmitter). The Raspberry Pi successfully reads, decrypts, interprets, and displays the sensor data, including displaying color swatches for the RGB and light intensity sensor readings.
Conclusion
In this post, we explored the use of the LoRa and LoRaWAN protocols to transmit environmental sensor data from an IoT device to an IoT gateway. Given its low energy consumption, long-distance transmission capabilities, and well-developed protocols, LoRaWAN is an ideal long-range wireless protocol for IoT devices.
This blog represents my own viewpoints and not of my employer, Amazon Web Services (AWS). All product names, logos, and brands are the property of their respective owners.