Instead of putting it in a public directory you could create an HTTP Handler which allows you to do authorization.
You can create an handler either by compiling and registering in web.config or by creating an ashx page. Then your page would authorize the user, and then write the contents to the response.
If you need session state be sure to inherit from IRequiresSessionState.
Edit
I'm a C# guy so here's a sample in that:
public class DownloadFile : IHttpHandler, System.Web.SessionState.IRequiresSessionState
{
public void ProcessRequest (HttpContext context) {
//Validate user is authenticiated and authorized.
if (context.Request.IsAuthenticated)
{
//This is your own custom authorization mechanism
if (AuthorizeUser(context.Request.User))
{
//not sure what the correct content type is. This is probally wrong
context.Response.ContentType = "application/xls";
//Setting size is optional
context.Response.AddHeader("Content-Disposition",
"attachment; filename=" + downloadName + "; size=" + myExcellFile.Length.ToString());
context.Response.BinaryWrite(myExcellFile);
}
}