Data Logging with a Raspberry Pi

A major limitation to the PC board I created for my ESP-WROOM-02 module described in the previous post was that it could only support one digital input (even though the ESP module itself supports many).  This was done intentionally to keep things simple.  In a real-world application, however, this would not be very useful.  Therefore, I decided to go with a different approach for round two.

I chose a Raspberry Pi 3B+ with battery backup to monitor, control and display everything.  This small but powerful platform would allow me to gather data from multiple sensors with plenty of capacity for future expansion.  The actual task of gathering sensor information and then processing it involved installing four pieces of software onto the RPi:

1) Collecting the data from each wireless sensor - I chose Aedes to serve as my MQTT broker/server.  MQTT is a messaging protocol ideally suited for the Internet of Things.  Devices connected to the service (clients) can transmit data (publish) and/or receive data (subscribe).  The broker essentially acts like a post office, handling all messages between devices while providing a level of security.  Note - at the moment only one of my sensors is wireless (a motion detector), while the rest (temperature and humidity) are hard wired directly to the Raspberry Pi via an I2C interface so no MQTT messaging is required.

2) Data manipulation - Node Red was my choice for handling all data once it was received.  It is essentially a visual programming tool consisting of nodes (units of pre-developed code) that perform a specific task such as writing to a database, converting raw data into a visual gauge or bar chart, filtering data, and so on.  Nodes are connected via virtual wires to create a flow.  The following image is part of a flow I created specifically for this project:

Node Red flow

In this example, the reddish-brown node in the center retrieves the hard wired sensor data (temperature and humidity) for the outside and basement sensors, the light orange nodes perform some post retrieval processing and the light brown nodes on the far right forward all processed data to the database software for storage.

3) Database storage and 4) Visualization - I decided upon InfluxDB for storing all my sensor data and Grafana for the user display.  InfluxDB is a time series database that allows for easy data logging and when coupled with additional software like Grafana provides a user friendly way to visualize data in real time.  The screenshot below shows my basement temperature and humidity over a 24hr period along with maximum, minimum and average values:

Grafana dashboard

A final touch was to create a simple web page that could display this information in real time using the browser on my smartphone, home computer, etc.  Node Red made adding this feature a snap.  Below is a screenshot of the display (I chose to show temperature only to keep things simple):

To access the page I just type in the IP Address of the Raspberry Pi (assigned by my WiFi router on initial startup).  Code for the gauge templates came from "canvas-gauges.com" but everything else was written by me.  This setup works fine when I'm around the house in range of our WiFi network but what about when I'm not at home?  The solution was to create a VPN (virtual private network) so that I could access this info anywhere in the world that has internet access.  I simply setup the RPi as a VPN server using a YouTube tutorial I found and it was ready to go.

Below is a picture of the Raspberry Pi (lower left) along with the battery charger (from DFRobot) and a +12V sealed lead acid battery enclosed in a plastic storage container kept in the basement.  The wires connected to the RPi go to my environmental sensors.

Update (Spring 2022)
I decided to completely overhaul my webpage by adding a few more bells and whistles as seen in the screenshot below.  The current temperature and rainfall values still come from my onsite sensors, but the 12-hour forecast readings come from an AccuWeather API.  I also added two temperature charts: one to show the hourly values throughout the day and one to show the daily low and high values for the week.  Finally, I added a background image to provide an interesting backdrop.  This came from a picture taken on one of our hikes near the Asheville area.