Programmatic Ponderings

Use a start-up script to overcome limitations of Motion/FFmpeg and save multiple Raspberry Pi dashboard camera timelapse videos and images, automatically.

Introduction

In my last post, Raspberry Pi-Powered Dashboard Video Camera Using Motion and FFmpeg, I demonstrated how the Raspberry Pi can be used as a low-cost dashboard video camera. One of the challenges I faced in that post was how to save the timelapse videos and individual images (frames) created by Motion and FFmpeg when the Raspberry Pi is turned on and off. Each time the car starts, the Raspberry Pi boots up, and Motion begins to run, the previous images and video, stored in the default ‘/tmp/motion/’ directory are removed and new images and video, created.

Take the average daily commute, we drive to and from work. Maybe we stop for a morning coffee, or stop at the store on the way home to pick up dinner. Maybe we use our car go out for lunch. Our car starts, stops, starts, stops, starts, and stops. Our daily commute actually encompasses a series small trips, and therefore multiple dash-cam timelapse videos. If you are only interested in keeping the latest timelapse video in case of an accident, then this may not be a problem. When the accident occurs, simply pull the SDHC card from the Raspberry Pi and copy the video and images off to your laptop.

However, if you are interested in capturing and preserving series of dash-cam videos, such as in the daily commute example above, then the default behavior of Motion is insufficient. To preserve each video segment or series of images, we need a way to preserve the content created by Motion and FFmpeg, before they are overwritten. In this post, I will present a solution to overcome this limitation.

The process involves the following steps:

1. Change the default location where Motion stores timelapse videos and images to somewhere other than a temporary directory;
2. Create a startup script that will move the video and images to a safe location when restarting the Pi;
3. Configure the Pi’s Debian operating system to run this script at startup (and optionally shutdown), before Motion starts;

Sounds pretty simple. However, understanding how startup scripts work with Debian’s Init program, making sure the new move script run before Motion starts, and knowing how to move a huge number of files, all required forethought.

Change Motion’s Default Location for Video and Images

To start, change the default location where Motion stores timelapse video and images, from ‘/tmp/motion/’ to a location outside the ‘/tmp’ directory. I chose to create a directory called ‘/motiontmp’. Make sure you set the permissions on the new ‘/motiontmp’ directory, so Motion can write to it:

sudo chmod -R 777 /motiontmp

To have Motion  use this location, we need to modify the Motion configuration file:

sudo nano /etc/motion/motion.conf

Change the following setting (in bold below). Note when Motion starts for the first time, it will create the ‘motion’ sub folder inside ‘motiontmp’. You do not have to create it yourself.

# Target base directory for pictures and films
# Recommended to use absolute path. (Default: current working directory)
target_dir /motiontmp/motion

Create the Startup Script to Move Video and Images

Next, create the new shell script that will run at startup to move Motion’s video and images. The script creates a timestamped folder in new ‘motiontmp’ directory for each series of images and video. The script then copies all files from the ‘motion’ directory to the new timestamped directory. Before copying, the script deletes any zero-byte jpegs, which are images that did not fully process prior to the Raspberry Pi being shut off when the car stopped. To create the new script, run the following command.

sudo nano /etc/init.d/motionStartup.sh

Copy the following contents into the script and save it.

### BEGIN INIT INFO
# Provides:          motionStartup
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Move motion files at startup.
# Description:       Move motion files at startup.
# X-Start-Before:    motion
### END INIT INFO

#! /bin/sh
# /etc/init.d/motionStartup
#

# Some things that run always
#touch /var/lock/motionStartup
logger -s "Script motionStartup called"

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    logger -s "Script motionStartup started"
    TIMESTAMP=$(date +%Y%m%d%H%M%S | sed 's/ //g') # No spaces
    logger -s "Script motionStartup $TIMESTAMP"
    sudo mkdir /motiontmp/$TIMESTAMP || logger -s "Error mkdir start"
    find /motiontmp/motion/. -type f -size 0 -print0 -delete
    find /motiontmp/motion/. -maxdepth 1 -type f | \
        xargs -I '{}' sudo mv {} /motiontmp/$TIMESTAMP
    ;;
  stop)
    logger -s "Script motionStartup stopped"
    ;;
  *)
    echo "Usage: /etc/init.d/motionStartup {start|stop}"
    exit 1
    ;;
esac

exit 0

Note the ‘X-Start-Before’ setting at the top of the script (in bold). An explanation of this setting is found on the Debian Wiki website. According to the site, ”There is no such standard-defined header, but there is a proposed extension implemented in the insserv package (since version 1.09.0-8). Use the X-Start-Before and X-Stop-After headers proposed by SuSe.” To make sure you have a current version of ‘insserv‘, you can run the following command:

dpkg -l insserv

Also, note how the files are moved by the script:

find /motiontmp/motion/. -maxdepth 1 -type f | \
        xargs -I '{}' sudo mv {} /motiontmp/$TIMESTAMP

It’s not as simple as using ‘mv *.*’ when you have a few thousand files. This will likely throw a ‘Argument list too long’ exception. According to one stackoverflow, the exception is because bash actually expands the asterisk to every matching file, producing a very long command line. Using the ‘find’ combined with ‘xargs’ gets around these problem. The ‘xargs’ command splits up the list and issues several commands if necessary. This issue applies to several commands, including rm, cp, and mv.

Lastly, note the use of the ‘logger‘ commands throughout the script. These are optional and may be removed. I like to log the script’s progress for troubleshooting purposes. Using the above ‘logger’ commands, I can easily pinpoint issues by looking at the log with grep, such as:

tail -500 /var/log/messages | grep 'motionStartup' | grep 'logger:'

You can test the script by running the following command:

/etc/init.d/./motionStartup.sh start

You should see a series of three messages output to the screen by the script, confirming the script is working. Note the new timestamped folder created by the script, below.

Below, is an example of the how the directory structure should look after a few videos are created by Motion, and the Raspberry Pi cycled off and on. You need to complete the rest of the steps in this post for this to work automatically.

Shutdown Script?

I know, the name of the post clearly says ‘Startup Script’. Well, a little tip, if you copy the code from the ‘start’ method and paste it in the ‘stop’ method, this now also works at shutdown. If you do a proper shutdown (like ‘sudo reboot’), the Raspberry Pi’s OS will call the script’s ‘stop’ method. The ‘start’ method is more useful to use for us in the car, where we may not be able to do a proper shutdown; we just turn the car off and kill power to the Pi. However, if you are shutting down from mobile device via ssh, or using a micro keyboard and LCD monitor, the script will do it’s work on the way down.

Configure Debian OS to Run the New Startup Script

To have our new script run on startup, install it by running the following command:

sudo update-rc.d motionStartup.sh defaults

A full explanation of this command is to complex for this brief post. A good overview of creating startup scripts and installing them in Debian is found on the Debian Administration website. This is the source I used to start to understand runlevels. There are also a few links at the end of the post. To tell which runlevel (state) you running at, use the following command:

runlevel

To make sure the startup script was installed properly, run the following command. This will display the contents of each ‘rc*.d’ folder. Each folder corresponds to a runlevel – 0, 1, 2, etc. Each folder contains symbolic links to the actual scripts. The links are named by order of executed (S01…, S02…, S03…):

ls /etc/rc*.d

Look for the new script listed under the appropriate runlevel(s). The new script should be listed before ‘motion’, as shown below.

If for any reason you need to uninstall the new script (not delete/remove), run the following command. This not a common task, but necessary to change the order of execution of the scripts or rename a script.

sudo update-rc.d -f motionStartup.sh remove

Copy and Remove Files from the Raspberry Pi

Once the startup script is working and we are capturing images and timelapse video, the next thing we will probably want to do is copy files off the Raspberry Pi. To do this over your WiFi network, use a ‘scp’ command from a remote machine. The below script copies all directories, stating with ‘2013’, and their contents to remote machine, preserving the directory structure.

scp -rp user@ip_address_of_pi:/motiontmp/2013* ~/local_directory/

Maybe you just want all the timelapse videos Motion/FFmpeg creates; you don’t care about the images. The following command copies just the MPEG videos from all ‘2013’ folders to a single directory on the your remote machine. The directory structure is ignored during the copy. This is the quickest way to just store all the videos.

scp -rp user@ip_address_of_pi:/motiontmp/2013*/*.mpg ~/local_directory/

If you are going to save all the MPEG timelapse videos in one location, I recommend changing the naming convention of the videos in the motion.conf file. I have added the hour, minute, and seconds to mine. This will ensure the names don’t conflict when moved to a common directory:

# File path for timelapse mpegs relative to target_dir
# Default: %Y%m%d-timelapse
# Default value is near equivalent to legacy oldlayout option
# For Motion 3.0 compatible mode choose: %Y/%m/%d-timelapse
# File extension .mpg is automatically added so do not include this
timelapse_filename %Y%m%d%H%M%S-timelapse

To remove all the videos and images once they have been moved off the Pi and are no longer needed, you can run a rm command. Using the ‘-rf’ options make sure the directories and their contents are removed.

sudo rm -rf /motiontmp/2013*

Conclusion

The only issue I have yet to overcome is maintaining the current time on the Raspberry Pi. The Pi lacks a Real Time Clock (RTC). Therefore, turning the Pi on and off in the car causes it to loose the current time. Since the Pi is not always on a WiFi network, it can’t sync to the current time when restarted. The only side-effects I’ve seen so far caused by this, the videos occasionally contain more than one driving event and the time displayed in the videos is not always correct. Otherwise, the process works pretty well.

Resources

The following are some useful resources on this topic:

Debian Reference: Chapter 3. The system initialization

How-To: Managing services with update-rc.d

Files and scripts that execute on boot

Making scripts run at boot time with Debian

Finding all files and move to new directory from shell prompt

Shell scripting: Write message to a syslog / log file

“Argument list too long”: Beyond Arguments and Limitations

Linux / Unix Command: date (used for TIMESTAMP)

Time / Date Commands

To have our new script run on startup, install it by running the following command: