Screen capture during testing

Question

This is an application for reviewing performance tests. Simple in concept, tricky to describe. Picture:

1) Recording interactions with a WPF program so the inputs can be played back.

2) Playing the inputs back while doing a continuous screen capture.

3) Capturing wall time as well as continuous CPU percentages during playback.

4) Repeating steps (2) and (3) lots of times.

5) Writing the relevant stuff out to files/db.

6) Reading it and putting it all in a fancy UI for easy review/analysis.

The killer for me is (2). I could use some guidance on a good, possibly commercial, screen capture SDK. I would also welcome the news that my whole problem already has a solution. And of course any thoughts on the overall idea would also be great.

Thanks.

Ed

Answer 1

Windows API? Maybe System.Drawing?

http://www.codeproject.com/KB/dialog/screencap.aspx

Answer 2

You can do the screen capture in your own application using DirectShow.

See Video Capture for general details on how to do video capture using DirectShow and links to example code. To capture specifically from the screen you'll need the "Screen Capture filter" filter found in wmpsrcwp.dll, which is found in newer versions of Windows and can also be downloaded as part as the Windows Media Encoder.

Although UI Automation can be used to play back events, it cannot record. If you need to record system events, I recommend you use Windows Hooks for both recording and playback.

DirectShow is easiest to do in C++/CLR instead of C# because you can easily include C++ header files and make unmanaged calls, and Windows Hooks should only be done in native code.

Answer 3

Are you creating a performance testing product or are you testing your product for performance?

If the latter, you could look at the feature set for Team Foundation Server 2010, specifically the testing tools. There seem to be some nice features there for historical debugging, profiling and QA team integration (test session video playback, etc).

Otherwise, if you're just looking for vidcap tool, I've always liked SnagIt.

Answer 4

If you are going to do development for this, you can setup Cucumber / SpecFlow using Windows Automation... and here's a sample in WPF of taking a screenshot of the App under test.

        /// <summary>
        /// Take screen shot
        /// </summary>
        /// <param name="left">left</param>
        /// <param name="top">top</param>
        /// <param name="width">width</param>
        /// <param name="height">height</param>
        /// <returns>screen shot in bytes</returns>
        public static byte[] TakeScreenShot(int left, int top, int width, int height)
        {
            // Set the bitmap object to the size of the screen
            var bmpScreenshot = new Bitmap(width, height,
                                                  PixelFormat.Format32bppArgb);

            // Create a graphics object from the bitmap
            var gfxScreenshot = Graphics.FromImage(bmpScreenshot);

            // Take the screenshot from the upper left corner to the right bottom corner
            gfxScreenshot.CopyFromScreen(left, top, 0, 0,
                                             new Size(width, height), CopyPixelOperation.SourceCopy);

            var buffer = new byte[bmpScreenshot.Size.Height * bmpScreenshot.Size.Width * 4];

            var stream = new MemoryStream(buffer);

            bmpScreenshot.Save(stream, ImageFormat.Png);

            return stream.ToArray();
        }

Answer 5

It sounds to me like you're looking for an automated testing tool that can execute UI interaction. We use TestComplete which I believe will do a lot of the functional UI testing and recording you're looking for. There are others as well. I'm wondering if you actually need to record the screen as the test is being run if you can easily reproduce the same interaction if something fails.

Answer 6

Since you're looking at commercial for screen capture, I'd recommend looking at commercial for the entire suite: I'd recommend Microsoft's Test Manager 2010 (see features here: http://msdn.microsoft.com/en-us/library/bb385901.aspx). It takes videos and records and plays back input, which is the bulk of what I can tell you're working on, but it also hooks right into .NET applications for intellitrace debugging purposes, integrates with Team Foundation Server's bug reporting system, etc. You can use it with WPF, XNA, ASP.Net, or other languages if you're willing to sacrifice the intellitrace capabilities. It'll even automate the playback of inputs and verify outputs to your specification for continuous integration testing.

Answer 7

You say the hard part for you is:

2) Playing the inputs back while doing a continuous screen capture.

You might consider using an automation and scripting tool such as AutoIt or AutoHotKey. Each of these tools will let you record your interactions with a Windows application. You might record a basic scenario, and then later adjust details and timing in the editor. A nice feature of AutoIt is the ability to "compile" your macro to a standalone executable.

As far as I can tell, neither of these tools will capture video. For that you might think about something like Fraps, which is a video screen grabbing made popular by video gamers. You should be able to use either of the above scripting tools to start and stop Fraps.

Good luck with your project.