Capturing HTML generated from ASP.NET

Question

How do I best capture the HTML (in my instance, for logging) rendered by an aspx-page?

I dont want to have to write back to the page using Response.Write, since it messes up my site layout.

Using the Response.OutputStream or Response.Output's stream results in an ArgumentException ({System.ArgumentException: Stream was not readable.)

Answer 1

Many load testers will allow you to log the HTTP responses generated, but bear in mind with ASP.NET those could be some very large log-files.

Edit: Response.Filter as per Tom Jelen's code is designed to give this kind of oversight and Response.Outputstream is otherwise unreadable.

Edit 2: For a page rather than a HTTPModule

public class ObserverStream : Stream
{
  private byte[] buffer = null;
  private Stream observed = null;

  public ObserverStream (Stream s)
  {
    this.observed = s;
  }

  /* important method to extend #1 : capturing the data */
  public override void Write(byte[] buffer, int offset, int count)
  {
    this.observed.Write(buffer, offset, count);
    this.buffer = buffer; //captured!
  }

  /* important method to extend #2 : doing something with the data */
  public override void Close()
  {
    //this.buffer available for logging here!
    this.observed.Close();
  }

  /* override all the other Stream methods/props with this.observed.method() */

  //...

}

and in your Page_Load (or before your response is written anyway)

Response.Filter = new ObserverStream(Response.Filter);

Answer 2

One way to to make server-side XMLHTTP request to your own server. Grab the result and save it to a file or DB.

Alternately you can use AJAX on the client, grab the result, and POST it back to the server.

Answer 3

Edit:

I need to stop editing this answer. To put it short, override the Render method for the page. Similar question and answers here

Answer 4

Good question, i had to try out and see if i could create a HttpModule to do what you are describing.

I didnt have any luck trying to read from the responsestream, but using the ResponseFilter gave me a way to capture the content.

The following code seems to work pretty good, and i figured maybe you could use the code as a base. But remember this is just something i threw together fast, it has not been tested in any way. So dont use it in any production environment without proper reviewing/testing and such. Feel free to comment on it though ;)

public class ResponseLoggerModule : IHttpModule
{
 private class ResponseCaptureStream : Stream
 {
  private readonly Stream _streamToCapture;
  private readonly Encoding _responseEncoding;

  private string _streamContent;
  public string StreamContent
  {
   get { return _streamContent; }
   private set
   {
    _streamContent = value;
   }
  }

  public ResponseCaptureStream(Stream streamToCapture, Encoding responseEncoding)
  {
   _responseEncoding = responseEncoding;
   _streamToCapture = streamToCapture;

  }

  public override bool CanRead
  {
   get { return _streamToCapture.CanRead; }
  }

  public override bool CanSeek
  {
   get { return _streamToCapture.CanSeek; }
  }

  public override bool CanWrite
  {
   get { return _streamToCapture.CanWrite; }
  }

  public override void Flush()
  {
   _streamToCapture.Flush();
  }

  public override long Length
  {
   get { return _streamToCapture.Length; }
  }

  public override long Position
  {
   get
   {
    return _streamToCapture.Position;
   }
   set
   {
    _streamToCapture.Position = value;
   }
  }

  public override int Read(byte[] buffer, int offset, int count)
  {
   return _streamToCapture.Read(buffer, offset, count);
  }

  public override long Seek(long offset, SeekOrigin origin)
  {
   return _streamToCapture.Seek(offset, origin);
  }

  public override void SetLength(long value)
  {
   _streamToCapture.SetLength(value);
  }

  public override void Write(byte[] buffer, int offset, int count)
  {
   _streamContent += _responseEncoding.GetString(buffer);
   _streamToCapture.Write(buffer, offset, count);
  }

  public override void Close()
  {
   _streamToCapture.Close();
   base.Close();
  }
 }

 #region IHttpModule Members

 private HttpApplication _context;
 public void Dispose()
 {

 }

 public void Init(HttpApplication context)
 {
  _context = context;

  context.PreRequestHandlerExecute += new EventHandler(context_PreRequestHandlerExecute);
  context.PreSendRequestContent += new EventHandler(context_PreSendRequestContent);
 }

 void context_PreRequestHandlerExecute(object sender, EventArgs e)
 {
  _context.Response.Filter = new ResponseCaptureStream(_context.Response.Filter, _context.Response.ContentEncoding);
 }

 void context_PreSendRequestContent(object sender, EventArgs e)
 {
  ResponseCaptureStream filter = _context.Response.Filter as ResponseCaptureStream;

  if (filter != null)
  {
   string responseText = filter.StreamContent;

   // Logging logic here
  }
 }

 #endregion
}