http client blocks on recv()

Question

Hi, I need some help writing an http client. The trouble comes when I try to receive data from a webserver. The recv() call blocks the program. Any better direction would be extremely helpful, I'll post my code below:

if ( argc != 2 )
{
 cerr << "Usage: " << argv[0];
 cerr << " <URI>" << endl;
 return 1;
}
else
{
 uri_string = argv[1];
}

// Create URI object and have it parse the uri_string
URI *uri = URI::Parse(uri_string);

if ( uri == NULL )
{
 cerr << "Error: Cannot parse URI." << endl;
 return 2;
}

// Check the port number specified, if none use port 80
unsigned port = 80;
if ( uri->Is_port_defined() )
{
 port = uri->Get_port();
}

// Create TCP socket and connect to server
int tcp_sock = socket( AF_INET, SOCK_STREAM, 0 );
if ( tcp_sock < 0 )
{
 cerr << "Unable to create TCP socket." << endl;
 return 3;
}

sockaddr_in server;
socklen_t slen = sizeof(server);

server.sin_family = AF_INET;
server.sin_port = htons( port );
hostent *hostp = gethostbyname( uri->Get_host().c_str() );
memcpy( &server.sin_addr, hostp->h_addr, hostp->h_length );

if ( connect( tcp_sock, (sockaddr*)&server, slen ) < 0 )
{
 cerr << "Unable to connect to server via TCP." << endl;
 close( tcp_sock );
 return 4;
}

// Build HTTP request to send to server
HTTP_Request *request = HTTP_Request::Create_GET_request( uri->Get_path() );
request->Set_host( uri->Get_host() );
string request_string = "";
request->Print( request_string );

//cout << request_string << endl;

// Send it to the server, wait for reply and use HTTP_Response to get reply
send( tcp_sock, &request_string, sizeof(request_string), 0 );

char recv_buffer[1024];
int bytes_recv = 0;
while ( bytes_recv < 1024 )
{
 int recv_len = recv( tcp_sock, recv_buffer + bytes_recv,
  1024 - bytes_recv, 0 );
 if ( recv_len == -1 )
 {
  cerr << "Error receiving response from server." << endl;
  close( tcp_sock );
  return 5;
 }
 bytes_recv += recv_len;
}


HTTP_Response *response = HTTP_Response::Parse(recv_buffer, bytes_recv);
string response_string = "";
response->Print( response_string );
cout << response_string << endl;

return 0;

}

Answer 1

Is this a problem?

If the client is comamnd line fine.
If it is GUI then the thread retrieving data should be different from the UI thread.

But a solution is to use the select()
This will tell you if there is anything to be read from a port.
Thus allowing you to do other work while waiting.

Answer 2

recv() is supposed to block until it gets a response. Are you sure you're writing your request properly, and that the server is responding to it? It's possible to put the file descriptor into nonblocking mode and test it using select() or poll(), but my guess is that you simply have a protocol bug somewhere. What is the behavior you are expecting?

Answer 3

The HTTP request should end with an empty line, i.e.

---

GET / HTTP/1.1
Host: blah.com
                    <- this here is an empty line

---

It looks like your code is not doing that (it should probably say *request->Print(request_string + "\n")*.

Off-topic: you know there are readily available HTTP clients in C, right? (such as libcurl).

Answer 4

You are using a blocking TCP/IP socket, but you are not looking at the HTTP reply's "Content-Length" header to know how many bytes to read. Your current reading logic is calling recv() in a loop until 1024 bytes max have been received. If the server sends less than 1024 bytes, you are going to be blocked indefinately because you are calling recv() too many times asking for too many bytes.

Answer 5

You must receive many bytes what is in content-length field. You must change line: if ( recv_len == -1 ) to:

if ( recv_len <= 0 ) break; else if ( recv_len == -1 )

because 0 is when server will disconnect after sending all data.

www.marsi.eu