For a project I’m working I needed a http client. Erlang has a couple like gun or hackney. However, I’ve noticed projects like erlcloud and rebar3 use httpc. If there are other http client libraries why should I use httpc? After some searching, I found this post on Erlang Forums. It seems that httpc is worth considering. So I decided to give it a go. Turns out it was a little more complicated so I thought a blog post walking through how to use it would be helpful. I’ll start with a basic version and progressively get to a more production ready form of it.

Basic Version

We will start with this simple function:

request(Method, Uri) ->
    request(Method, Uri, Headers).

request(Method, Uri, Headers) ->
    do_request(Method, {Uri, Headers}).

request(Method, Uri, Headers, ContentType, Body) ->
   do_request(Method, {Uri, Headers, ContentType, Body}).

do_request(Method, Request) ->
    handle_response(httpc:request(
        Method, 
        Request,
        [{ssl, [{verify, verify_none}]}],
        []
    )).

handle_response(Resp) ->
    case Resp of
        {ok, {_, RespHeaders, RespBody}} ->
            {ok, RespHeaders, RespBody};
        {error, _Reason} = E -> E
    end.

This will work for basic stuff but it’s susceptible to MITM attacks because we don’t do any verification. This behaviour is what we want for making requests to http like to localhost.

A basic GET request might look like this:

request(get, "http://localhost:8080/todos").

Maybe the API requires an Authorization header:

request(get, "http://localhost:8080/todos", [{"Authorization", "Bearer secretkey"}]).

To make a POST request with a JSON body is slightly more complicated:

request(
    post, 
    "http://localhost:8080/todo"
    [{"Content-Type", "application/json"}],
    "application/json",
    <<"{\"body\":\"Buy Groceries\"}">>
).

Notice we had to set the Content-Type header and tell httpc the content type as well. A bit weird, but whatever we can make it work. You could change the function to check for the Content-Type header and just use that. This works fine though.

Secure Version

Now we need to make this function secure. This turns out to be a little more complicated than I expected.

First we need to add the certifi library as a dependency.

Then we will make a helper function:

ssl_opts() ->
    {ssl, [
        {verify, verify_peer},
        {cacertfile, certifi:cacertfile()},
        {customize_hostname_check, [{match_fun, public_key:pkix_verify_hostname_match_fun(https)}]}.

These options will make the connection verify the response is coming from the requested host.

Let’s use this helper function in our do_request function:

do_request(Method, Request) ->
    handle_response(httpc:request(
      Method,
      {Uri, Headers},
      [ssl_opts()],
      Body
    )).

Now we can make requests across the web in a secure way.

request(get, "https://example.com").

However, we can’t make requests to our localhost. What if we need to both?

Final Version

Let’s modify the ssl_opts function to take the URI and check if it is http or https.

Luckily Erlang has a builtin url parser function that will help with this.

We will need another helper function to get the protocol from the URL.

get_protocol(URL) ->
    case uri_string:parse(URL) of
        {ok, Parsed} ->
            maps:get(scheme, Parsed, undefined);
        _Error ->
            undefined
    end.

Let’s use this function as part of ssl_opts.

ssl_opts(URL) ->
    Options =
    case get_protocol(URL) of
        "https" ->
            [{verify, verify_peer},
             {cacertfile, certifi:cacertfile()},
             {customize_hostname_check, [{match_fun, public_key:pkix_verify_hostname_match_fun(https)}]}];
        _ ->
            [{verify, verify_none}]
    end,
    {ssl, Options}.

Basically we use the verification options if it is https and if its anything else we don’t verify. Now we can use the same request function for https or http.

With all this you should be set to get started with httpc and make your own http requests.