any suggestion on clustered client server comm. approach?

Question

Let's say i'm building a client server software. the main server connects to agents on different servers. the agents is a search client and each will return quite huge array of 64bit integers and finally the main server (parent) will sort it to produce finaly result.

Which is better/faster approach,

1. using a socket to get all the data from client, or
2. use socket to send client state and request id, the result then stored on temp flat file with that id (main server will then need to access that temp file located on client svr)?

Answer 1

I wouldn't use a temp a temp file on the client since accessing this over the network won't be any faster then writing it's contents directly over a socket to the server. You might however want to design your protocol to send the data in smaller pieces for which you get ack responses from server so that you don't need to resend the entire data if your connection breaks during the send. Another advantage of this is that you let's the server cancel the transmission by response execpt by closing the connection if it doesn't want it anymore for some reason.

Answer 2

KISS principle is your friend here.

If your goal is performance - then you have to do it with sockets and store everything in memory

If you want to make it fast to implement and simple - I would just go with simple HTTP requests, and would put files on server's shared disk (I would say, it's 3 times faster to implement).