Lately I’ve been interested in learning about satellites and got inspired to make a simple IoT project. I can’t make my own satellite, but I could at least make a device to take periodic pictures. My idea was that since I’m not up to see the sunrise I would have this bot take pictures out my window in the morning. This project turned out to be both easier and harder than I initially thought, and it was fun learning experience.

While it might’ve been easier to use Python this felt like a perfect use case for Erlang as an IoT project. Also, I love Erlang. So I’ll go through the process I went through to get a Raspberry Pi setup and made an Erlang project to acheive my goal.

The code that I’ll be discussing can be found here.

Setting up the Pi

Since I felt like making my life harder I decided to try out the officially supported Debian Raspberry images instead of Raspbian. The nice that thing is that is a very minimal image with less of the cruft baked into Raspbian. The downside is that docs are a little sparse. It took me a little bit of time to setup wifi, but that wasn’t a surprise…

I downloaded the image from here. Once downloaded the image needs to be copied to the SD card. I used this command:

xzcat raspi_${RPI_MODEL}_${DEBIAN_RELEASE}.img.xz | dd of=${SD_CARD} bs=64k oflag=dsync status=progress

Since ssh isn’t setup yet you’ll need an HDMI adapter and keyboard to plug into the pi to do the next parts.

Login as root. And there is no password. I’ll leave it as an exercise for the reader to change the root password. Once you are logged in you will be able to setup wifi.

Wifi on the Debian Pi

After sometime scouring the web I stumbled on this article.

The basic steps are editing /etc/network/interfaces.d/wlan0. To look like this:

allow-hotplug wlan0
iface wlan0 inet dhcp
    wpa-ssid my-network-ssid
    wpa-psk s3kr3t_P4ss

One thing I also did was disable IPV6. (I don’t remember why, but I had it in my notes. So use your best judgement.)

Setup the software

Once wifi is connected you can install the needed dependencies.

apt update && apt upgrade
apt install erlang rebar3 g++ make git libopencv-dev

One thing I noticed about the erlang package in Debian is it installs a bunch of things you don’t need. I would recommend the erlang-core package instead.

Setting up the Camera

I had a spare USB webcam that I decided to use for this project instead of buying anything special. It was more than enough to get going and provided an interesting starting point. Here are the steps I took to get the webcam working with Debian.

adduser user
usermod -a -G video user

I found a useful CLI tool to check for video devices.

apt install v4l-utils
v4l2-ctl --list-devices

If you can’t list devices with your user you likely need to be added to the video group. When sshed in, make sure to end the session and start a new one for the video permissions to show.

OpenCV Code

The OpenCV code was written in C++ since that is how it is provided as a library. I tried unsuccessfully to use the C api.

I was able to write all the native code in one file that looked like this:

#include "erl_nif.h"
#include <iostream>
#include <chrono>
#include <thread>
#include <opencv2/opencv.hpp>

using namespace cv;
using namespace std;

...

static ERL_NIF_TERM
capture_pic(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[])
{

...

        Mat frame;
        VideoCapture cap;
        cap.open(device);


        if (!cap.isOpened()) {
                cerr << "ERROR: Unable to open camera" << endl;
                cap.release();
                return mk_error(env, "camera_error");
        }


        if (!(res_width == 0) && !(res_height == 0)) {
                // Set resolution
                cap.set(CAP_PROP_FRAME_WIDTH, res_width);
                cap.set(CAP_PROP_FRAME_HEIGHT, res_height);
        }

        std::this_thread::sleep_for(std::chrono::milliseconds(5000));
        cap.read(frame);
        cap.release();

        bool result = false;
        try {
                result = imwrite(buf, frame);
        } catch (const cv::Exception& ex) {
                fprintf(stderr, "Exception converting image to PNG format: %s\n", ex.what());
        }

        if (!result) {
                return mk_error(env, "opencv_write_error");
        }
        return mk_atom(env, "ok");
    }

The capture_pic function opens the camera device, sets the resolution, sleeps for 5 seconds, takes the picture and then writes out the image buffer to disc.

Makefile

I made a nif with the rebar3 nif generator. Mostly everything in the code is from that. The two lines I needed to change were these:

Since the OpenCV code is C++, we need to add the dependency for CXXFLAGS:

CXXFLAGS += -fPIC -I $(ERTS_INCLUDE_DIR) -I $(ERL_INTERFACE_INCLUDE_DIR) `pkg-config --cflags opencv4`

Then the same thing for linking the library:

LDLIBS += -L $(ERL_INTERFACE_LIB_DIR) -lei `pkg-config --libs opencv4` 

Solving Camera Problems

While I had my code mostly working I ran into a couple issues with the physical camera device. This was making thankful for software because at least I get error messages there! Here I was just getting crappy pictures.

The first issue was not giving the camera enough time to turn on and let the lighting adjusted. This was an easy fix by adding a sleep in the C++ code after camera initialization.

std::this_thread::sleep_for(std::chrono::milliseconds(5000));

The second issue I ran into that turned out to be common with OpenCV was not setting the resolution. Otherwise by default it will use the minimum resolution. Otherwise the image will have artifacts. The code to fix this looks like this:

if (!(res_width == 0) && !(res_height == 0)) {
        // Set resolution
        cap.set(CAP_PROP_FRAME_WIDTH, res_width);
        cap.set(CAP_PROP_FRAME_HEIGHT, res_height);
}

Dealing with Networking issues

One requirement I felt like adding was to upload the pictures to S3 instead of keeping them locally. This was partially because a Raspberry PI doesn’t have infinite storage space and the pictures need to go somewhere. With this requirement adds networking. Surprisngly, despite this being on my home wifi I ran into a few issues. Also, one thing I’ve learned over the years is to never trust S3 uploads completely. This creates the need for retry logic.

Need to have resiliency to network outages because at minimum can get timeouts with uploading because of slow raspbi. In the instance of an upload failure the backup plan is to keep the image on disk and then try to reupload periodically.

Found a stack overflow with almost exact problem and use case.

Remove power management for wifi interface on the RPi:

apt install wireless-tools net-tools

iwconfig wlan0 power off

Reboot to see effect. Probably not great if you have limited power but I have it plugged in to a power supply.

Erlang Code

You may have noticed that this post is titled “erlpic” yet there has been no Erlang. Let’s fix that.

The NIF wrapping the C++ code showed earlied looks like this:

erlpic_nif:capture_pic(binary_to_list(Path), Device, ResWidth, ResHeight)

Once the image is captured it is written to disk which needs to be uploaded. I used the erlcloud library for the s3 upload.

erlcloud_s3:put_object("erlpic", filename:basename(Path), Binary)

Here we see that the bucket being uploaded to is erlpic, the object key is the name of the file excluding the file path, and the binary of the image.

The function I used to cleanup files after successful uploads was this:

lists:foreach(fun(X) -> file:delete(X) end, absolute_paths(PrivDir, Filenames2));

At a high level this logic is wrapped in a single function that looks like this:

capture() ->
    case capture_pic() of
        {ok, Path} ->
            upload_pic(Path);
        {error, _Err} = E ->
            E
    end.

We take the picture and we upload it. Simple.

This gives us most of the functionality we need, but we still need to run this on a timer. Lucky for us, Erlang has a common pattern for this.

We can use a GenServer (the basic OTP primitive for a process), that will create a timer on startup. The code for this is quite small so I’ll post most of it here:

-module(erlpic_server).

-export([start_link/0, init/1, handle_info/2]).

-define(INTERVAL, 60000 * 5). %% 5 Minutes

start_link() ->
    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

-spec init(list()) -> {ok, list()}.
init(_Args) ->
    erlang:send_after(10000, ?MODULE, trigger),
    {ok, []}.

-spec handle_info(trigger | any(), any()) -> {noreply, any()}.
handle_info(trigger, State) ->
    trigger(),
    {noreply, State};
handle_info(_Msg, State) ->
    {noreply, State}.

-spec trigger() -> reference().
trigger() ->
    capture(),
    sync_remaining(),
    erlang:send_after(?INTERVAL, ?MODULE, trigger).

The key function call is this:

erlang:send_after(10000, ?MODULE, trigger)

Which means that Erlang will send an async message to the ?MODULE (which is a macro that will be translated to erlpic_server) after 10 seconds. That message only needs to contain the atom trigger, so it will be matched with the handle_info function definition. That will call the trigger function, it will capture an image, sync any remaining images that didn’t successfully upload, and start the cycle over again but with the ?INTERVAL constant set at the top of the module.

All of this is compiled into a release with rebar3 release. This needed to be run on the pi because it uses native code and the pi runs on ARM, whereas my laptop is x86.

systemd

Now I wanted this process to be a systemd daemon to start at boot. This prevents me from needing to ssh and restart the process as well as just being the expected way to make a daemon.

[Unit]
Description=Picture taking service
After=network.target

[Service]
ExecStart=/bin/sh -lc '/home/pi/erlpic/_build/default/rel/erlpic/bin/erlpic foreground'
Type=simple
User=pi
Restart=always

[Install]
WantedBy=multi-user.target

One issue I ran into was the daemon not inheriting the AWS credentials I set in ~/.profile. Needed to adjust ExecStart to start with a login shell. Notice that the ExecStart is calling the Erlang release that is compiled earlier.

Misc

The enif_inspect_binary wouldn’t be NULL terminated. This resulted in invalid file paths for OpenCV to write to.

Running code in the init method of a gen_server throws an error that will crash the entire application. A better solution is to decouple it by sending a message in the init method.

Overall this was a fun project and if you got this far hopefully you learned something as I know I did.