Posts Tagged Rasp Pi

Using a Startup Script to Save Motion/FFmpeg Videos and Images on The Raspberry Pi

Use a start-up script to overcome limitations of Motion/FFmpeg and save multiple Raspberry Pi dashboard camera timelapse videos and images, automatically.

01-20130624190454-00

Introduction

In my last post, Raspberry Pi-Powered Dashboard Video Camera Using Motion and FFmpeg, I demonstrated how the Raspberry Pi can be used as a low-cost dashboard video camera. One of the challenges I faced in that post was how to save the timelapse videos and individual images (frames) created by Motion and FFmpeg when the Raspberry Pi is turned on and off. Each time the car starts, the Raspberry Pi boots up, and Motion begins to run, the previous images and video, stored in the default ‘/tmp/motion/’ directory are removed and new images and video, created.

Take the average daily commute, we drive to and from work. Maybe we stop for a morning coffee, or stop at the store on the way home to pick up dinner. Maybe we use our car go out for lunch. Our car starts, stops, starts, stops, starts, and stops. Our daily commute actually encompasses a series small trips, and therefore multiple dash-cam timelapse videos. If you are only interested in keeping the latest timelapse video in case of an accident, then this may not be a problem. When the accident occurs, simply pull the SDHC card from the Raspberry Pi and copy the video and images off to your laptop.

However, if you are interested in capturing and preserving series of dash-cam videos, such as in the daily commute example above, then the default behavior of Motion is insufficient. To preserve each video segment or series of images, we need a way to preserve the content created by Motion and FFmpeg, before they are overwritten. In this post, I will present a solution to overcome this limitation.

The process involves the following steps:

  1. Change the default location where Motion stores timelapse videos and images to somewhere other than a temporary directory;
  2. Create a startup script that will move the video and images to a safe location when restarting the Pi;
  3. Configure the Pi’s Debian operating system to run this script at startup (and optionally shutdown), before Motion starts;

Sounds pretty simple. However, understanding how startup scripts work with Debian’s Init program, making sure the new move script run before Motion starts, and knowing how to move a huge number of files, all required forethought.

Change Motion’s Default Location for Video and Images

To start, change the default location where Motion stores timelapse video and images, from ‘/tmp/motion/’ to a location outside the ‘/tmp’ directory. I chose to create a directory called ‘/motiontmp’. Make sure you set the permissions on the new ‘/motiontmp’ directory, so Motion can write to it:

sudo chmod -R 777 /motiontmp

To have Motion  use this location, we need to modify the Motion configuration file:

sudo nano /etc/motion/motion.conf

Change the following setting (in bold below). Note when Motion starts for the first time, it will create the ‘motion’ sub folder inside ‘motiontmp’. You do not have to create it yourself.

# Target base directory for pictures and films
# Recommended to use absolute path. (Default: current working directory)
target_dir /motiontmp/motion

Motion Target Directory

Create the Startup Script to Move Video and Images

Next, create the new shell script that will run at startup to move Motion’s video and images. The script creates a timestamped folder in new ‘motiontmp’ directory for each series of images and video. The script then copies all files from the ‘motion’ directory to the new timestamped directory. Before copying, the script deletes any zero-byte jpegs, which are images that did not fully process prior to the Raspberry Pi being shut off when the car stopped. To create the new script, run the following command.

sudo nano /etc/init.d/motionStartup.sh

Copy the following contents into the script and save it.

### BEGIN INIT INFO
# Provides:          motionStartup
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Move motion files at startup.
# Description:       Move motion files at startup.
# X-Start-Before:    motion
### END INIT INFO

#! /bin/sh
# /etc/init.d/motionStartup
#

# Some things that run always
#touch /var/lock/motionStartup
logger -s "Script motionStartup called"

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    logger -s "Script motionStartup started"
    TIMESTAMP=$(date +%Y%m%d%H%M%S | sed 's/ //g') # No spaces
    logger -s "Script motionStartup $TIMESTAMP"
    sudo mkdir /motiontmp/$TIMESTAMP || logger -s "Error mkdir start"
    find /motiontmp/motion/. -type f -size 0 -print0 -delete
    find /motiontmp/motion/. -maxdepth 1 -type f | \
        xargs -I '{}' sudo mv {} /motiontmp/$TIMESTAMP
    ;;
  stop)
    logger -s "Script motionStartup stopped"
    ;;
  *)
    echo "Usage: /etc/init.d/motionStartup {start|stop}"
    exit 1
    ;;
esac

exit 0

Note the ‘X-Start-Before’ setting at the top of the script (in bold). An explanation of this setting is found on the Debian Wiki website. According to the site, ”There is no such standard-defined header, but there is a proposed extension implemented in the insserv package (since version 1.09.0-8). Use the X-Start-Before and X-Stop-After headers proposed by SuSe.” To make sure you have a current version of ‘insserv‘, you can run the following command:

dpkg -l insserv

Also, note how the files are moved by the script:

find /motiontmp/motion/. -maxdepth 1 -type f | \
        xargs -I '{}' sudo mv {} /motiontmp/$TIMESTAMP

It’s not as simple as using ‘mv *.*’ when you have a few thousand files. This will likely throw a ‘Argument list too long’ exception. According to one stackoverflow, the exception is because bash actually expands the asterisk to every matching file, producing a very long command line. Using the ‘find’ combined with ‘xargs’ gets around these problem. The ‘xargs’ command splits up the list and issues several commands if necessary. This issue applies to several commands, including rm, cp, and mv.

Lastly, note the use of the ‘logger‘ commands throughout the script. These are optional and may be removed. I like to log the script’s progress for troubleshooting purposes. Using the above ‘logger’ commands, I can easily pinpoint issues by looking at the log with grep, such as:

tail -500 /var/log/messages | grep 'motionStartup' | grep 'logger:'

View of Log with Script Messages

You can test the script by running the following command:

/etc/init.d/./motionStartup.sh start

You should see a series of three messages output to the screen by the script, confirming the script is working. Note the new timestamped folder created by the script, below.

Testing the New Script

Below, is an example of the how the directory structure should look after a few videos are created by Motion, and the Raspberry Pi cycled off and on. You need to complete the rest of the steps in this post for this to work automatically.


Shutdown Script?

I know, the name of the post clearly says ‘Startup Script’. Well, a little tip, if you copy the code from the ‘start’ method and paste it in the ‘stop’ method, this now also works at shutdown. If you do a proper shutdown (like ‘sudo reboot’), the Raspberry Pi’s OS will call the script’s ‘stop’ method. The ‘start’ method is more useful to use for us in the car, where we may not be able to do a proper shutdown; we just turn the car off and kill power to the Pi. However, if you are shutting down from mobile device via ssh, or using a micro keyboard and LCD monitor, the script will do it’s work on the way down.

Configure Debian OS to Run the New Startup Script

To have our new script run on startup, install it by running the following command:

sudo update-rc.d motionStartup.sh defaults

A full explanation of this command is to complex for this brief post. A good overview of creating startup scripts and installing them in Debian is found on the Debian Administration website. This is the source I used to start to understand runlevels. There are also a few links at the end of the post. To tell which runlevel (state) you running at, use the following command:

runlevel

To make sure the startup script was installed properly, run the following command. This will display the contents of each ‘rc*.d’ folder. Each folder corresponds to a runlevel – 0, 1, 2, etc. Each folder contains symbolic links to the actual scripts. The links are named by order of executed (S01…, S02…, S03…):

ls /etc/rc*.d

Look for the new script listed under the appropriate runlevel(s). The new script should be listed before ‘motion’, as shown below.

View of Runlevels

View of Runlevels 2

If for any reason you need to uninstall the new script (not delete/remove), run the following command. This not a common task, but necessary to change the order of execution of the scripts or rename a script.

sudo update-rc.d -f motionStartup.sh remove

Copy and Remove Files from the Raspberry Pi

Once the startup script is working and we are capturing images and timelapse video, the next thing we will probably want to do is copy files off the Raspberry Pi. To do this over your WiFi network, use a ‘scp’ command from a remote machine. The below script copies all directories, stating with ‘2013’, and their contents to remote machine, preserving the directory structure.

scp -rp user@ip_address_of_pi:/motiontmp/2013* ~/local_directory/

Maybe you just want all the timelapse videos Motion/FFmpeg creates; you don’t care about the images. The following command copies just the MPEG videos from all ‘2013’ folders to a single directory on the your remote machine. The directory structure is ignored during the copy. This is the quickest way to just store all the videos.

scp -rp user@ip_address_of_pi:/motiontmp/2013*/*.mpg ~/local_directory/

If you are going to save all the MPEG timelapse videos in one location, I recommend changing the naming convention of the videos in the motion.conf file. I have added the hour, minute, and seconds to mine. This will ensure the names don’t conflict when moved to a common directory:

# File path for timelapse mpegs relative to target_dir
# Default: %Y%m%d-timelapse
# Default value is near equivalent to legacy oldlayout option
# For Motion 3.0 compatible mode choose: %Y/%m/%d-timelapse
# File extension .mpg is automatically added so do not include this
timelapse_filename %Y%m%d%H%M%S-timelapse

To remove all the videos and images once they have been moved off the Pi and are no longer needed, you can run a rm command. Using the ‘-rf’ options make sure the directories and their contents are removed.

sudo rm -rf /motiontmp/2013*

Conclusion

The only issue I have yet to overcome is maintaining the current time on the Raspberry Pi. The Pi lacks a Real Time Clock (RTC). Therefore, turning the Pi on and off in the car causes it to loose the current time. Since the Pi is not always on a WiFi network, it can’t sync to the current time when restarted. The only side-effects I’ve seen so far caused by this, the videos occasionally contain more than one driving event and the time displayed in the videos is not always correct. Otherwise, the process works pretty well.

Resources

The following are some useful resources on this topic:

Debian Reference: Chapter 3. The system initialization

How-To: Managing services with update-rc.d

Files and scripts that execute on boot

Making scripts run at boot time with Debian

Finding all files and move to new directory from shell prompt

Shell scripting: Write message to a syslog / log file

“Argument list too long”: Beyond Arguments and Limitations

Linux / Unix Command: date (used for TIMESTAMP)

Time / Date Commands

To have our new script run on startup, install it by running the following command:

, , , , , , , , , ,

3 Comments

Installing TightVNC on the Raspberry Pi

Sometimes connecting a keyboard, mouse, and monitor to Raspberry Pi is really inconvenient. But what’s the alternative if you want to interact directly with your Raspberry Pi’s GUI? PuTTY is an excellent SSH client, but the command shell is no substitute. WinSCP is an excellent SFTP client, but again, no substitute for a fully-functional GUI. The answer to this predicament? TightVNC, by GlavSoft LLC.

Background

According to TightVNC Software’s website, ‘TightVNC is a free remote control software package. With TightVNC, you can see the desktop of a remote machine and control it with your local mouse and keyboard, just like you would do it sitting in the front of that computer.

What is VNC? According to Wikipedia, ‘Virtual Network Computing (VNC) is a graphical desktop sharing system that uses the RFB protocol (remote framebuffer) to remotely control another computer. It transmits the keyboard and mouse events from one computer to another, relaying the graphical screen updates back in the other direction, over a network.

If you are a Windows user, you are no doubt familiar with Microsoft’s Remote Desktop Connection (RDC). GlavSoft’s TightVNC and Microsoft’s RDC are almost identical in terms of functionality.

Installation

TightVNC has two parts, the client and the server. The TightVNC Server software is installed on the Raspberry Pi (RaspPi). The RaspPi acts as the TightVNC Server. The client software, the TightVNC Java Viewer, is installed on a client laptop or desktop computer.

I used PuTTY from my Windows 8 laptop to perform the following installation and configuration. I successfully performed this process on a RaspPi Model B, with copies of both Raspbian “wheezy” and Soft-float Debian “wheezy”.

TightVNC Server
To install the TightVNC Server software, run the following commands from the RaspPi. The first command is are optional, but usually recommended before installing new software.

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install tightvncserver

To test the success of the TightVNC Server installation, enter ‘vncserver‘ in the command shell. The first time you run this command, you will be asked to set a VNC password for the current user (‘pi’). The password can be different than the system password used by this user. After inputting a password, you should see output similar to the below screen grab. This indicates that TightVNC is running.

First Time vcnserver Command 1

By default, TightVNC runs on a port 5901. To verify TightVNC is running on 5901, enter the command ‘sudo netstat -tulpn‘. You should see output similar to the screen grab below. Note the entry for TightVNC on port 5901. Stop TightVNC by entering the ‘vncserver -kill :1‘ command.

First Time vcnserver Command 2

You may have noticed TightVNC was also running on port 6001. This is actually used by the X Window System, aka ‘X11’. A discussion of X11 is out of scope for this post, but more info can be found here.

Automatic Startup
For TightVNC Server to start automatically when we boot up our RaspPi, we need to create an init script and add it to the default runlevels. I had a lot of problems with this part until I found this post, with detailed instructions on how to perform these steps.

Start by entering the following command to create the init script:

sudo nano /etc/init.d/tightvncserver

Copy and paste the init script from the above post, into this file. Change the user from ‘pi’ to your user if it is different than ‘pi’. Save and close the file.

Next, execute these two commands to add the script to the default runlevels:

sudo chmod 755 /etc/init.d/tightvncserver
sudo update-rc.d tightvncserver defaults

To complete the TightVNC Server installation, restart the RaspPi.

TightVNC Java Viewer
According to the website, TightVNC Java Viewer is a fully functional remote control client written entirely in Java. It can work on any computer where Java is installed. It requires Java SE version 1.6 or any later version. That can be Windows or Mac OS, Linux or Solaris — it does not make any difference. And it can work in your browser as well. On the client computer, download and unzip the TightVNC Java Viewer. At the time of this post, the current TightVNC Java Viewer version was 2.6.2.

Once the installation is complete, double-click on the ‘tightvnc-jviewer.jar’ file. Running the Java jar file will bring up the ‘New TightVNC Connection’ window, as seen in the example below. Input the RaspPi’s IP address or hostname, and the default TightVNC port of 5901. The use of SSH tunneling is optional with the TightVNC Viewer. If you are concerned about security, use SSH.

TightVNC Connection Window

Clicking the ‘Connect’ button, you are presented with a window to input the user’s VNC password.

TightVNC VNC Authentication Window

Optionally, if using SSH, the user’s SSH password is required. Again, the same user can have different SSH and VNC passwords, as mine does.

TightVNC SSH Authentication Window

If everything was installed and configured correctly, you should be presented with a TightVNC window displaying the RaspPi’s desktop. Note the TightVNC toolbar along the top edge of the window. The ‘Ctrl’ and ‘Alt’ buttons are especially useful to send either of these two key inputs to the RaspPi on a Windows client. Using the ‘Set Options’ button, you can change the quality of TightVNC’s remote display. Note these changes this can affect performance.

Raspberry Pi's X Desktop

Congratulations, no more connecting a keyboard, mouse, and monitor to you RaspPi to access the GUI. I suggest reading the documentation on the TightVNC website, as well as the ‘README.txt’ file, included with the TightVNC Java Viewer. There is a lot more to TightVNC than I have covered in this brief introductory post, especially in the README.txt file. -gs

, , , , , , , , , , , , , , ,

11 Comments