Home Automation with Raspberry Pi

From Wurst-Wasser.net
Jump to: navigation, search

The Why Of Fry[1]

I moved to a house and would like to monitor and automate some things:

  • log room temperatures
  • log outside temperature
  • provide a webcam
  • save the world :)

Preparation

Parts list

  • Raspberry Pi (I formerly used this Rev 2 for http://www.wurst-wasser.net/wiki/index.php/Tomat-O-Mat - I have a garden now and don’t need it anymore)
  • Power supply for the Pi
  • DHT11 (measures temperature and humidity but really sucks because it's badly supported)
  • DS18B20 (measures temperature)
  • Raspi camera
  • some resistors, cables, wires, boards, and the usual soldering equipment

Setting up the Pi

Since the Pi was running Debian Wheezy (which is AFAIK obsolete) I decided to install Raspbian Stretch.

I skipped NOOBS and did the installation myself (with help from https://www.raspberrypi.org/documentation/installation/installing-images/README.md and https://www.raspberrypi.org/documentation/installation/installing-images/mac.md ):

# diskutil list
[…]
/dev/disk5 (external, physical):
  #:                       TYPE NAME                    SIZE       IDENTIFIER
  0:     FDisk_partition_scheme                        *15.8 GB    disk5
  1:             Windows_FAT_32                         58.7 MB    disk5s1
  2:                      Linux                         15.8 GB    disk5s2
# diskutil unmountDisk /dev/disk5
Unmount of all volumes on disk5 was successful
# dd bs=1m if=2018-11-13-raspbian-stretch-full.img of=/dev/rdisk5 conv=sync
5052+0 records in
5052+0 records out
5297405952 bytes transferred in 486.102151 secs (10897722 bytes/sec)
# diskutil eject /dev/rdisk5
Disk /dev/rdisk5 ejected
#

After that, enable interfaces (ssh, camera port or 1-wire) as needed (either using GUI or Raspi-config).

Update the Software

Use the graphical interface or some CLI (apt) tool. Setting up WiFi might help :)

Set up your shell language

I strongly recommend keeping your shell (or complete OS) in english.

Install Tools

# dpkg --configure -a
# apt-get install locate
# apt-get install gnuplot
# apt-get install git
# apt-get install apache2
# apt-get install php
# apt-get install imagemagick

DHT11 - Humidity & Temperature sensor

Setting up the DHT11 1-wire device

RaspberryPiDHT11.png
Remember that the Pi revisions have different pinouts! This will work for Revison 2, that is board revision 0004[2]. If you don't need humidity readings, I strongly recommend using a DS18B20 instead of the DHT11, as showed in RaspberryPi Temperature Sensor, because you can use the 1-wire kernel modules for reading the DS18B20, which makes it much easier to read and extend.

Installing DHT11 driver

Since there seems to be no kernel module for DHT11, do this:

# cd /home/pi/bin
# git clone git://git.drogon.net/wiringPi
# cd wiringPi/
# git pull origin
# ./build

Create a file hum_temp3.c with this content (Code taken from here (Rahul Kar) and here):

/* from http://www.circuitbasics.com/how-to-set-up-the-dht11-humidity-sensor-on-the-raspberry-pi/ 

Based on hum_temp2.c. Reads data until it gets a valid set. Then prints it once.

*/

#include <wiringPi.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#define MAXTIMINGS	85
#define DHTPIN		7
int dht11_dat[5] = { 0, 0, 0, 0, 0 };

int read_dht11_dat(char *message)
{
   uint8_t laststate	= HIGH;
   uint8_t counter		= 0;
   uint8_t j		= 0, i;
   float	f; 
       int			err=0;

   dht11_dat[0] = dht11_dat[1] = dht11_dat[2] = dht11_dat[3] = dht11_dat[4] = 0;

   pinMode( DHTPIN, OUTPUT );
   digitalWrite( DHTPIN, LOW );
   delay( 18 );
   digitalWrite( DHTPIN, HIGH );
   delayMicroseconds( 40 );
   pinMode( DHTPIN, INPUT );

   for ( i = 0; i < MAXTIMINGS; i++ )
   {
       counter = 0;
       while ( digitalRead( DHTPIN ) == laststate )
       {
           counter++;
           delayMicroseconds( 1 );
           if ( counter == 255 )
           {
               break;
           }
       }
       laststate = digitalRead( DHTPIN );

       if ( counter == 255 )
           break;

       if ( (i >= 4) && (i % 2 == 0) )
       {
           dht11_dat[j / 8] <<= 1;
           if ( counter > 16 )
               dht11_dat[j / 8] |= 1;
           j++;
       }
   }

   if ( (j >= 40) &&
       (dht11_dat[4] == ( (dht11_dat[0] + dht11_dat[1] + dht11_dat[2] + dht11_dat[3]) & 0xFF) ) )
   {
       f = dht11_dat[2] * 9. / 5. + 32;
       sprintf(message, "Humidity: %d.%d%%, Temperature: %d.%dC (%.1fF)\n",
           dht11_dat[0], dht11_dat[1], dht11_dat[2], dht11_dat[3], f );
               err=0;
   }
       else
       {
           sprintf(message, "Data not good, skip\n" );
               err=1;
   }

 return (err);
}

int main( void )
{
   //printf( "Raspberry Pi wiringPi DHT11 Temperature test program\n" );

   if ( wiringPiSetup() == -1 ) exit( 1 );

       char message[1024];
       int error=1;
       int tries=0;
   while ( error!=0 && tries < 42 )
   {
       tries++;
       error=read_dht11_dat(message);
       delay( 1000 ); 
   }

       if (error == 0)
       {
          printf(message);
       }
       else
       {
          printf("Failed to read data (%i tries)", tries);
       }

   return(0);
}


Compile it:

# gcc -o sensor3 hum_temp3.c -L/usr/local/lib -lwiringPi

Test it:

./sensor3
Humidity: 35.0%, Temperature: 22.0C (71.6F)

readHumidityAndTemperature.sh

#/bin/bash
#set -x
#
# Script: readHumidityAndTemperature.sh
# Author: Heiko Kretschmer
# Purpose: Reading the humidity/temperature and generating a nice plot
#
#

#
# Globals
#
GBASENAME="readHumidityAndTemperature"
GTIMESTAMPFORMAT="%Y-%m-%d_%H:%M"
GTIMESTAMP="`date +${GTIMESTAMPFORMAT}`"
GTIMESTAMPTIME="`date '+%H:%M'`"
GTIMESTAMPDATE="`date '+%Y-%m-%d'`" # ISO 8601 date format
GTIMESTAMPDATEHUMANREADABLE="`date '+%A, %Y-%m-%d'`"
#GFOLDERBASE="/home/pi/var/humidity_and_temperature"
GFOLDERBASEVAR="/home/pi/var/humidity_and_temperature"
GFOLDERBASEBIN="/home/pi/bin/"
GCMDSENSOR="${GFOLDERBASEBIN}/wiringPi/sensor3" # prints "Humidity: 35.0%, Temperature: 22.0C (71.6F)"
GFOLDERLOGS="${GFOLDERBASEVAR}/logs"
GFOLDERGRAPHS="${GFOLDERBASEVAR}/graphs"
GFOLDERTMP="${GFOLDERBASEVAR}/tmp"
GFILELOGHUM="${GFOLDERLOGS}/${GBASENAME}_${GTIMESTAMPDATE}_Humidity.log"
GFILELOGTEMP="${GFOLDERLOGS}/${GBASENAME}_${GTIMESTAMPDATE}_Temperature.log"
GFILEGRAPHHUM="${GFOLDERGRAPHS}/${GTIMESTAMPDATE}_Humidity.svg"
GFILEGRAPHTEMP="${GFOLDERGRAPHS}/${GTIMESTAMPDATE}_Temperature.svg"
GFILEGRAPHHUMTEMP="${GFOLDERGRAPHS}/${GTIMESTAMPDATE}_Humidity_Temperature.svg"
GFILEPLOTCOMMANDS="${GFOLDERTMP}/${GBASENAME}-plot.cmd"
GTIMESTAMPFORMAT="%Y-%m-%d_%H:%M"
GTIMESTAMP="`date +${GTIMESTAMPFORMAT}`"
GTIMESTAMPTIME="`date '+%H:%M'`"
GTIMESTAMPMINUTES="`date +'%M'`"
GTIMESTAMPHOURS="`date +'%H'`"
GUPLOADEVERYTIME=0
GFILELOCK="${GFOLDERBASEVAR}/${GBASENAME}.lck"
GFILELOCKAGEMAX="60"
GFOLDERWWW="/var/www/html/webcam"
GFOLDERWWWARCHIVE="/var/www/html/webcam-archive"

#
# Functions 
#
function log
{
   logger "${GBASENAME}: $1"
   echo $1
} 


FPLOTGRAPHDOUBLE() # Plotting a single graph into one diagramm
{
  DATAFILE1=$1
  LINE1COLOUR=$2
  LINE1LABEL=$3
  LINE1UNITS=$4
  DATAFILE2=$5
  LINE2COLOUR=$6
  LINE2LABEL=$7
  LINE2UNITS=$8
  OUTFILE=$9

  test -f "${GFILEPLOTCOMMANDS}" && rm "${GFILEPLOTCOMMANDS}"

  # Reset
  echo "reset" >> "${GFILEPLOTCOMMANDS}"

  # Set output file type to svg
  echo "set term svg size 640,480" >> "${GFILEPLOTCOMMANDS}"
  echo "set output \"${OUTFILE}\"" >> "${GFILEPLOTCOMMANDS}"
  
  # Key
  echo "set key inside right top vertical Right noreverse enhanced autotitles columnhead nobox" >> "${GFILEPLOTCOMMANDS}"

  # Set time format for X axis
  echo "set timefmt \"%H:%M\"" >> "${GFILEPLOTCOMMANDS}"
  echo "set xdata time" >> "${GFILEPLOTCOMMANDS}"
  echo "set format x \"%H:%M\"" >> "${GFILEPLOTCOMMANDS}"

  # Setup line style (#1) for the first line
  echo "set style line 1 lc rgb \"${LINE1COLOUR}\" pt 1 ps 1 lt 1 lw 2" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/

  # Setup line style (#2) for the second line
  echo "set style line 2 lc rgb \"${LINE2COLOUR}\" pt 1 ps 1 lt 1 lw 2" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/
  echo "set style data lines" >> "${GFILEPLOTCOMMANDS}"

  # Set X tics (one tic per hour, rotate that tick-labels by 90 deg and move em a bit)
  echo "set xtics \"01:00\" rotate by 90 offset 0,-2 out nomirror" >> "${GFILEPLOTCOMMANDS}"

  # Setup Grid (with line style #12)
  echo "set style line 12 lc rgb '#E0E0E0' lt 0 lw 1" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/
  echo "set grid back ls 12" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/

  # Setup Title
  echo "set title \"${LINE1LABEL} and ${LINE2LABEL} on ${GTIMESTAMPDATEHUMANREADABLE}\"" >> "${GFILEPLOTCOMMANDS}"

  # Label X and Y Axis
  echo "set ylabel \"${LINE1UNITS}\"" >> "${GFILEPLOTCOMMANDS}"
  echo "set y2label \"${LINE2UNITS}\"" >> "${GFILEPLOTCOMMANDS}"
  echo "set xlabel \"Time\" offset 0,-0.5" >> "${GFILEPLOTCOMMANDS}"

  # Setup Y range(s)
  echo "set yrange [ 0.000 : 100.000 ] noreverse nowriteback" >> "${GFILEPLOTCOMMANDS}"
  echo "set y2range [ 0.000 : 50.000  ] noreverse nowriteback" >> "${GFILEPLOTCOMMANDS}"

  # Tics on the outside, also do the tics for y2, too
  echo "set tics out" >> "${GFILEPLOTCOMMANDS}"
  echo "set y2tics" >> "${GFILEPLOTCOMMANDS}"

  # Plot line 1
  echo "plot \"${DATAFILE1}\" using 1:2 title '${LINE1LABEL}' with l ls 1 axes x1y1, \"${DATAFILE2}\" using 1:2 title '${LINE2LABEL}' with l ls 2 axes x1y2" >> "${GFILEPLOTCOMMANDS}"

  cat "${GFILEPLOTCOMMANDS}" 
  cat "${GFILEPLOTCOMMANDS}" | gnuplot 

} # end of FPLOTGRAPHDOUBLE


FPLOTGRAPHSINGLE() # Plotting a single graph
{
  DATAFILE=$1
  LINE1COLOUR=$2
  LINE1LABEL=$3
  LINE1UNITS=$4
  OUTFILE=$5
  test -f "${GFILEPLOTCOMMANDS}" && rm "${GFILEPLOTCOMMANDS}"
  echo "reset" >> "${GFILEPLOTCOMMANDS}"
  echo "set key inside right top vertical Right noreverse enhanced autotitles columnhead nobox" >> "${GFILEPLOTCOMMANDS}"

  # Set time format for X axis
  echo "set timefmt \"%H:%M\"" >> "${GFILEPLOTCOMMANDS}"
  echo "set xdata time" >> "${GFILEPLOTCOMMANDS}"
  echo "set format x \"%H:%M\"" >> "${GFILEPLOTCOMMANDS}"

  # Setup line style (#1) for the temperature line
  echo "set style line 1 lc rgb \"${LINE1COLOUR}\" pt 1 ps 1 lt 1 lw 2" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/
  echo "set style data lines" >> "${GFILEPLOTCOMMANDS}"

  # Set X tics (one tic per hour, rotate that tick-labels by 90 deg and move em a bit)
  echo "set xtics \"01:00\" rotate by 90 offset 0,-2 out nomirror" >> "${GFILEPLOTCOMMANDS}"

  # Setup Grid (with line style #12)
  echo "set style line 12 lc rgb '#E0E0E0' lt 0 lw 1" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/
  echo "set grid back ls 12" >> "${GFILEPLOTCOMMANDS}" # http://www.gnuplotting.org/tag/grid/

  # Setup Title
  echo "set title \"${LINE1LABEL} on ${GTIMESTAMPDATEHUMANREADABLE}\"" >> "${GFILEPLOTCOMMANDS}"

  # Label X and Y Axis
  echo "set ylabel \"${LINE1UNITS}\"" >> "${GFILEPLOTCOMMANDS}"
  echo "set xlabel \"Time\" offset 0,-0.5" >> "${GFILEPLOTCOMMANDS}"

  # Setup Y range
  echo "set yrange [ 0.000 : ] noreverse nowriteback" >> "${GFILEPLOTCOMMANDS}"

  # Set output file type to svg and plot it into file
  echo "set term svg size 640,480" >> "${GFILEPLOTCOMMANDS}"
  echo "set output \"${OUTFILE}\"" >> "${GFILEPLOTCOMMANDS}"
  echo "plot \"${DATAFILE}\" using 1:2 title '${LINE1LABEL}' with l ls 1" >> "${GFILEPLOTCOMMANDS}"
  cat "${GFILEPLOTCOMMANDS}" | gnuplot 
}

#
# Main
#

if [ "${1}" = "-f" ]; then
  log "Forcing upload."
  GUPLOADEVERYTIME=1
fi

# Setup folders
mkdir -p "${GFOLDERBASEVAR}"

# Init
test ! -d "${GFOLDERLOGS}" && mkdir "${GFOLDERLOGS}"
test ! -d "${GFOLDERGRAPHS}" && mkdir "${GFOLDERGRAPHS}"
test ! -d "${GFOLDERTMP}" && mkdir "${GFOLDERTMP}"

# Check locking
if [ -f "${GFILELOCK}" ]; then 
  LOCKFILEAGE="`stat --format=\"%Z\" \"${GFILELOCK}\"`"
  NOW="`date +'%s'`"
  AGE="`expr ${NOW} - ${LOCKFILEAGE}`"
  if [ "${AGE}" -gt "${GFILELOCKAGEMAX}" ]; then
     log "Lockfile is ${AGE} seconds old, removing it."
     rm "${GFILELOCK}"
  else
     log "Lockfile is ${AGE} seconds old, exiting."
     exit 1
  fi
fi
#
#
# Lock
echo $$ > "${GFILELOCK}"

# Get values
LINE="`sudo ${GCMDSENSOR}`"
echo "LINE: ${LINE}"
HUMIDITY="`echo \"${LINE}\" | cut -d: -f2 | cut -d\\% -f1 | awk '{printf $1}'`"
echo "HUMIDITY: ${HUMIDITY}"
TEMPERATURE="`echo \"${LINE}\" | cut -d: -f3 | cut -dC -f1 | awk '{printf $1}'`"
echo "TEMPERATURE: ${TEMPERATURE}"
#exit 1

if [ "${HUMIDITY}" != "Invalid" -a "${TEMPERATURE}" != "Invalid" ]; then

  # Write it into the log (one logfile per day to get nice graphs)
  echo -e "${GTIMESTAMPTIME}\t${HUMIDITY}" >> "${GFILELOGHUM}"
  echo -e "${GTIMESTAMPTIME}\t${TEMPERATURE}" >> "${GFILELOGTEMP}"
#  echo -e "${GTIMESTAMPTIME}\t${HUMIDITY}\t${TEMPERATURE}" >> "${GFILELOGHUMTEMP}"

else
  echo "Read from sensor failed, plotting (old data) anyway."
fi

  # Generate the graphs (single)
  FPLOTGRAPHSINGLE "${GFILELOGHUM}" '#0e1a8b' "Humidity" "%" "${GFILEGRAPHHUM}"
  FPLOTGRAPHSINGLE "${GFILELOGTEMP}" '#8b1a0e' "Temperature" "°C" "${GFILEGRAPHTEMP}"

  # Generate combined graph (double)
  FPLOTGRAPHDOUBLE "${GFILELOGHUM}" '#0e1a8b' "Humidity" "%" "${GFILELOGTEMP}" '#8b1a0e' "Temperature" "°C" "${GFILEGRAPHHUMTEMP}"

  # Move Files
  ~pi/bin/moveGraphs.sh

  # Cleanup: Delete all files older than n*24 hours
  #find "${GFOLDERTARGET}" -ctime +1 -delete
  
  # Unlock
  rm "${GFILELOCK}"





# EOF

moveGraphs.sh

I moved the file transfers into a separate script. What you do with them depends on what you intend to do. I do:

  • move them to the local apache2 directory
  • upload it on my public webserver (using a chrooted user for the upload)
  • I also recommend using knockd

Cronjob

You might want to use a cronjob like this:

*/15 * * * *    /bin/bash /home/pi/bin/readHumidityAndTemperature.sh > /dev/null 2>&1

Setting up the camera

Installation

  • Power the Pi off
  • Plug the camera into the port next to the ethernet-port. The blue isolated side facing the ethernet-port.
  • boot your Pi
  • enable camera interface with…
raspi-config

Taking pictures

takePicture.sh

#!/bin/bash
#
# Globals
#
#GFOLDERTARGET="/var/www/html/webcam"
GFOLDERTARGET="/home/pi/var/webcam"
GFILELOCK="${GFOLDERTARGET}/takePicture.lck"
GFILELOCKAGEMAX="60"
GTIMESTAMPDATE="`date '+%Y-%m-%d'`" # ISO 8601 date format
GTIMESTAMPTIME="`date +'%H:%M:%S'`"
GTIMESTAMP="${GTIMESTAMPDATE}_${GTIMESTAMPTIME}"
GTIMESTAMPNICE="${GTIMESTAMPDATE}, ${GTIMESTAMPTIME}"

#
#
# Main
#

#
# Create output directory
# 
mkdir -p "${GFOLDERTARGET}"

#
# Check locking
#
if [ -f "${GFILELOCK}" ]; then
  LOCKFILEAGE="`stat --format=\"%Z\" \"${GFILELOCK}\"`"
  NOW="`date +'%s'`"
  AGE="`expr ${NOW} - ${LOCKFILEAGE}`"
  if [ "${AGE}" -gt "${GFILELOCKAGEMAX}" ]; then
     echo "Lockfile is ${AGE} seconds old, removing it."
     rm "${GFILELOCK}"
  else
     echo "Lockfile is ${AGE} seconds old, exiting."
     exit 1
  fi
fi
#
#
# Lock
echo $$ > "${GFILELOCK}"

#
# Take picture
#
# timeout should not be 0, since images will be to dark and are noisy
# Since my cam is mounted wrong, I prefer to flip it :) (and flop it) (flip: horizontal, flop: vertical)
raspistill --nopreview -w 1024 -h 768 -o - > "${GFOLDERTARGET}/${GTIMESTAMP}.tmp"
#raspistill --nopreview -w 1024 -h 768 -o - | convert - -flip -flop - > "${GFOLDERTARGET}/${GTIMESTAMP}.tmp"

#
# Blur it, if necessary (https://stackoverflow.com/questions/47064408/blur-part-of-image-in-imagemagick & http://www.imagemagick.org/Usage/mapping/#blur )
#
if [ 1 -eq 1 ]; then
   convert "${GFOLDERTARGET}/${GTIMESTAMP}.tmp" \( -clone 0 -fill white -colorize 100 -fill black -draw "polygon 0,400 1023,400 1023,767 0,767" -alpha off -write mpr:mask +delete \) -mask mpr:mask -blur 0x9 +mask "${GFOLDERTARGET}/${GTIMESTAMP}.tmp.blur"
   mv "${GFOLDERTARGET}/${GTIMESTAMP}.tmp.blur" "${GFOLDERTARGET}/${GTIMESTAMP}.tmp"
fi

#
# Imprint timestamp
#
mogrify -stroke yellow -pointsize 12 -gravity SouthEast -draw "text 0,0 '${GTIMESTAMPNICE}'" "${GFOLDERTARGET}/${GTIMESTAMP}.tmp"

#
# Move into üplace
#
mv "${GFOLDERTARGET}/${GTIMESTAMP}.tmp" "${GFOLDERTARGET}/${GTIMESTAMP}.jpg"

#
# Move & upload
#
~pi/bin/movePictures.sh

#
# Unlock
#
rm "${GFILELOCK}"

#
# Exit
#

# EOF

movePictures.sh

I moved the file transfers into a separate script. What you do with them depends on what you intend to do. I do:

  • move them to the local apache2 directory
  • upload it on my public webserver (using a chrooted user for the upload)
  • I also recommend using knockd

Cronjob

You might want to use a cronjob like this:

*/15 * * * *    /bin/bash /home/pi/bin/takePicture.sh > /dev/null 2>&1


Adding more Temperature Sensors (DS18B20)

I guess I'm f*d with this DHT11 - it uses the same breakout pins like the usual 1-wire devices. So there is (beside of some hack I'm too tired to perform) no way to connect this DS18B20 additionally)


Setting up

TBD

The Script

TBD

Setting up Watchdogs

The Raspberry Pis are very unreliable machines. Honestly, for what they do all day long the freeze a lot (sometimes more than once a week). This might help:



References: