Option 3 is the best:
Use Async IO.
Unless your request processing is complex and heavy, your program is going to spend 99% of it's time waiting for the HTTP requests.
This is exactly what Async IO is designed for - Let the windows networking stack (or .net framework or whatever) worry about all the waiting, and just use a single thread to dispatch and 'pick up' the results.
Unfortunately the .NET framework makes it a right pain in the ass. It's easier if you're just using raw sockets or the Win32 api. Here's a (tested!) example using C#3 anyway:
using System.Net; // need this somewhere
// need to declare an class so we can cast our state object back out
class RequestState
{
public WebRequest Request { get; set; }
}
static void Main( string[] args )
{
// stupid cast neccessary to create the request
HttpWebRequest request = WebRequest.Create( "http://www.stackoverflow.com" ) as HttpWebRequest;
request.BeginGetResponse(
(asyncResult) => { /* callback to be invoked when finished */
var state = (RequestState)asyncResult.AsyncState; // fetch the request object out of the AsyncState
var webResponse = state.Request.EndGetResponse( asyncResult ) as HttpWebResponse;
Debug.Assert( webResponse.StatusCode == HttpStatusCode.OK ); // there we go;
Console.WriteLine( "Got Response from server:" + webResponse.Server );
},
new RequestState { Request = request } /* pass the request through to our callback */ );
// blah
Console.WriteLine( "Waiting for response. Press a key to quit" );
Console.ReadKey();
}
EDIT:
In the case of .NET, the 'completion callback' actually gets fired in a ThreadPool thread, not in your main thread, so you will still need to lock any shared resources, but it still saves you all the trouble of managing threads.