views:

993

answers:

4

Sometimes boost::asio seems to disconnect before I want it to, i.e. before the server properly handles the disconnect. I'm not sure how this is possible because the client seems to think its fully sent the message, yet when the server emits the error its not even read the message header... During testing this only happens maybe 1 in 5 times, the server receives the client shut down message, and disconnects the client cleanly.

The error: "An existing connection was forcibly closed by the remote host"

The client disconnecting:

void disconnect()
{
    boost::system::error_code error;
    //just creates a simple buffer with a shutdown header
    boost::uint8_t *packet = createPacket(PC_SHUTDOWN,0);
    //sends it
    if(!sendBlocking(socket,packet,&error))
    {
        //didnt get here in my tests, so its not that the write failed...
        logWrite(LOG_ERROR,"server",
            std::string("Error sending shutdown message.\n")
            + boost::system::system_error(error).what());
    }

    //actaully disconnect
    socket.close();
    ioService.stop();
}
bool sendBlocking(boost::asio::ip::tcp::socket &socket,
    boost::uint8_t *data, boost::system::error_code* error)
{
    //get the length section from the message
    boost::uint16_t len = *(boost::uint16_t*)(data - 3);
    //send it
    asio::write(socket, asio::buffer(data-3,len+3),
        asio::transfer_all(), *error);
    deletePacket(data);
    return !(*error);
}

The server:

void Client::clientShutdown()
{
    //not getting here in problem cases
    disconnect();
}
void Client::packetHandler(boost::uint8_t type, boost::uint8_t *data,
    boost::uint16_t len, const boost::system::error_code& error)
{
    if(error)
    {
        //error handled here
        delete[] data;
        std::stringstream ss;
        ss << "Error recieving packet.\n";
        ss << logInfo() << "\n";
        ss << "Error: " << boost::system::system_error(error).what();
        logWrite(LOG_ERROR,"Client",ss.str());

        disconnect();
    }
    else
    {
        //call handlers based on type, most will then call startRead when
        //done to get the next packet. Note however, that clientShutdown
        //does not
        ...
    }
}



void startRead(boost::asio::ip::tcp::socket &socket, PacketHandler handler)
{
    boost::uint8_t *header = new boost::uint8_t[3];
    boost::asio::async_read(socket,boost::asio::buffer(header,3),
        boost::bind(&handleReadHeader,&socket,handler,header, 
        boost::asio::placeholders::bytes_transferred,boost::asio::placeholders::error));
}
void handleReadHeader(boost::asio::ip::tcp::socket *socket, PacketHandler handler,
    boost::uint8_t *header, size_t len, const boost::system::error_code& error)
{
    if(error)
    {
        //error "thrown" here, len always = 0 in problem cases...
        delete[] header;
        handler(0,0,0,error);
    }
    else
    {
        assert(len == 3);
        boost::uint16_t payLoadLen  = *((boost::uint16_t*)(header + 0));
        boost::uint8_t  type        = *((boost::uint8_t*) (header + 2));
        delete[] header;
        boost::uint8_t *payLoad = new boost::uint8_t[payLoadLen];

        boost::asio::async_read(*socket,boost::asio::buffer(payLoad,payLoadLen),
            boost::bind(&handleReadBody,socket,handler,
            type,payLoad,payLoadLen,
            boost::asio::placeholders::bytes_transferred,boost::asio::placeholders::error));
    }
}
void handleReadBody(ip::tcp::socket *socket, PacketHandler handler,
    boost::uint8_t type, boost::uint8_t *payLoad, boost::uint16_t len,
    size_t readLen, const boost::system::error_code& error)
{
    if(error)
    {
        delete[] payLoad;
        handler(0,0,0,error);
    }
    else
    {
        assert(len == readLen);
        handler(type,payLoad,len,error);
        //delete[] payLoad;
    }
}
+2  A: 

Maybe this is what is happening:

  • Client send disconnect packet
  • Client shuts socket down
  • Server read handler gets called, but there is an error associated with the shutdown packet because the socket is already closed.

I see in your read handlers, if there is an error, you never check to see if your shutdown packet is there. Maybe it is. Basically what I'm saying is maybe your client sometimes is able to send both the close and the shutdown packet before the server has a chance to process them separately.

Chris H
"//error "thrown" here, len always = 0 in problem cases..." so its always read 0 bytes of the header, ie it hasn't read any of the packet... And sticking a Sleep(500) or something on the client isn't a good solution, because that still may not always be enough on slower networks and is a noticeable delay.
Fire Lancer
Perhaps wait for an OK package from the server, or have the server disconnect instead?
Marcus Lindblom
But if the client then waits for the server to send a disconnect message (in response to the client disconnect message), I'll just end up with the opposite problem of the server disconnecting on the client....
Fire Lancer
Would you be able to use the disconnect error alone or do you really need the disconnect packet for something?
Chris H
A: 

I think you should probably have a call to socket.shutdown(boost::asio::ip::tcp::socket::shutdown_both, ec) in there before the call to socket.close().

The boost::asio documentation for basic_stream_socket::close states:

For portable behaviour with respect to graceful closure of a connected socket, call shutdown() before closing the socket.

This should ensure that any pending operations on the socket are properly cancelled and any buffers are flushed prior to the call to socket.close.

GrahamS
+1  A: 

Use async_write() and put socket.close() inside of write handler. This will make sure packet is processed by boost asio and not neglected in the middle of processing (because of close() calls).

flamemyst
A: 

I have a very similar issue. I believe it's related to Windows recycling connections. Is the following familiar?

  • you get this error immediately upon starting the program but not after a connection is established?
  • The error never happens if you wait more than 4 minutes before restarting your application?

The tcp specs specify that by default it should wait four minutes for the final acknowledgment when a tcp connection is closed. You can see these connections in FIN_WAIT state using netstat. The Windows OS detects when you try to connect to the exact same system and takes these partially closed connections and recycles them. Your second invocation of the program gets the 'closed' connection left behind by the first run. It gets the next acknowledge and then really closes.

Jay