Handling Erlang inets http client errors

Question

I have an Erlang app which makes a large number of http calls to external sites using inets, using the code below

case http:request(get, {Url, []}, [{autoredirect, false}], []) of
{ok, {{_, Code, _}, _, Body}}->
    case Code of
    200 ->
        HandlerFn(Body);
    _ ->
        {error, io:format("~s returned HTTP ~p", [Broker, Code])}
    end;
Response -> %% block to handle unexpected responses from inets
    {error, io:format("~s returned ~p", [Broker, Response])}
end.

There is an explicit block to handle anything strange inets might return [Response]. Despite this, I still get what look like inets error reports dumped to the console [sample below]. What am I doing wrong here ? Do I need to configure some kind of inets error handler elsewhere ?

Thanks.

--

=ERROR REPORT==== 24-Apr-2010::06:49:47 ===
** Generic server <0.6618.0> terminating 
** Last message in was {connect_and_send,
                           {request,#Ref<0.0.0.139358>,<0.6613.0>,0,http,
                               {"**********",80},
                               "*****************************",
                               [],get,
                               {http_request_h,undefined,"keep-alive",
                                   undefined,undefined,undefined,undefined,
                                   undefined,undefined,undefined,undefined,
                                   undefined,undefined,undefined,undefined,
                                   undefined,undefined,"news.bbc.co.uk",
                                   undefined,undefined,undefined,undefined,
                                   undefined,undefined,undefined,undefined,
                                   undefined,[],undefined,undefined,undefined,
                                   undefined,"0",undefined,undefined,
                                   undefined,undefined,undefined,undefined,[]},
                               {[],[]},
                               {http_options,"HTTP/1.1",infinity,false,[],
                                   undefined,false,infinity},
                               "************************************",
                               [],none,[],1272088179114,undefined,undefined}}
** When Server state == {state,
                            {request,#Ref<0.0.0.139358>,<0.6613.0>,0,http,
                                {"******************",80},
                                "*****************************",
                                [],get,
                                {http_request_h,undefined,"keep-alive",
                                    undefined,undefined,undefined,undefined,
                                    undefined,undefined,undefined,undefined,
                                    undefined,undefined,undefined,undefined,
                                    undefined,undefined,"news.bbc.co.uk",
                                    undefined,undefined,undefined,undefined,
                                    undefined,undefined,undefined,undefined,
                                    undefined,[],undefined,undefined,
                                    undefined,undefined,"0",undefined,
                                    undefined,undefined,undefined,undefined,
                                    undefined,[]},
                                {[],[]},
                                {http_options,"HTTP/1.1",infinity,false,[],
                                    undefined,false,infinity},
                                "****************************************",
                                [],none,[],1272088179114,undefined,undefined},
                            undefined,undefined,undefined,undefined,undefined,
                            {[],[]},
                            {[],[]},
                            undefined,[],nolimit,nolimit,
                            {options,
                                {undefined,[]},
                                0,2,5,120000,2,disabled,false,inet,default,
                                default,[]},
                            {timers,[],undefined},
                            httpc_manager,undefined}
** Reason for termination == 
** {error,{connect_failed,{#Ref<0.0.0.139358>,{error,nxdomain}}}}

=ERROR REPORT==== 24-Apr-2010::06:49:47 ===
HTTPC-MANAGER<httpc_manager> handler (<0.6618.0>, started) failed to connect and/or send request #Ref<0.0.0.139358>
   Result: {error,{connect_failed,{#Ref<0.0.0.139358>,{error,nxdomain}}}}

Answer 1

For each http request you make, a separate httpc_handler process is spawned "internally". This process first tries to open a socket to the desired domain. In this case the domain is non-existent, and so opening the socket fails. As a result, the spawned process decides to stop.

As the handler process is written according to the gen_server principles, your error handler will flush the last state of the dying process. There's nothing much you could or should do about this.

Answer 2

I'd guess that http creates a new process which dies with nxdomain. This crash sasl picks up and prints in the shell.

Answer 3

Assuming your app follows OTP's format then the supervisor handling the inets process should be configurable to restart the process. Crashing processes is "normal" for an erlang app and there are all sorts ways in a supervisor tree to handle the occasion.

Lukas was right in that SASL is merely reporting the fact that the process crashed. This prevents your callback from ever getting called so it can't handle the crash. In fact it can't handle the crash since it specifically depends on getting called by the process that crashed.

Without seeing how your inets process is started and the code that makes the call It's hard to advise you any more than to say that the proper place to handle the crash is in your supervisor tree. I would suggest reading up on the Erlang System Design documentation: http://www.erlang.org/doc/design_principles/users_guide.html to get a feel for how erlang processes are meant to be used and handled.

Answer 4

The inets HTTP client has a rather inconsistent interface. I would suggest using lhttpc instead.