Video Capturing + Uploading + Processing + Streaming back - .NET & C#

Question

We are trying to find out any technologies/libraries available in .NET stack (even wrappers on top of 3rd party dlls) that'll help us to build an app that can

• 1 - Capture an image from a user's video device
• 2 - Upload it realtime to a server
• 3 - Process the video (in the server) - eg: Adding a watermark to the video
• 4 - Stream it back to the user/other users

Preferably, the time delay/latency between step2 and 4 should be minimal

The first requirement (capturing) seems pretty straight forward. The challenge is identifying a suitable way to do the upload, do the processing, and stream it back. Any valid suggestions or ideas?

Recently came acrsoss FFmpeg library, and it has a C# wrapper. Does FFmpeg can be used to do the processing side?

Answer 1

It seems that Splicer can process static video and convert it - I'm not sure about processing a realtime uploaded video - http://splicer.codeplex.com/

Answer 2

VideoLab from Mitov can accomplish all of this and is free for personal use (not so free for commercial use, but pricing is not too heavy).

I have bought and use the Delphi version and know it works extremely well, so I'm pretty sure the .NET version will do what you need.

This kind of task is not trivial (as seen by the lack of responses here), so expect to struggle considerably with DirectX/Microsoft Media Encoder- but with this toolkit and some help from the author, you will eventually succeed.

http://www.mitov.com/html/videolab.html

Answer 3

The question is kind of short on details (is this a web server, what os is the server? etc) but I'll take a stab based on what I think you're trying to do.

One thing you might consider is doing the capture and process at one time. If the user is running your client app, have that do the capture and processing via DirectShow. Then all you need to do is upload the video and you can skip the entire server process. This is assuming that the 'user' is under your control - that this is not some random person out there uploading video, but an employee or someone otherwise trusted.

If this isnt the case, then ffmpeg can certainly be used to watermark video on your server. You dont really need 'wrappers' for it. You can just call it as a command line app from your server application and wait for it to finish.

The process really isnt that complex... its the details that are going to matter (for example - what does 'stream' mean to you? Do you really mean 'stream', or is this via http? Thats a huge topic right there)

Answer 4

Hi,

I would go about it this way:

1. Use silverlight or flash to capture the video camera input, e.g. as detailed here.
2. You can send the byte-stream over a socket that your server is listening to.
3. On the receiving end, just use the socket-accepting program as a router-program with a number of listening workers connected. Between workers and router-program, e.g. AMQP with RabbitMQ. Send asynchronous messages (e.g. with reactive extensions) with e.g. the stream encoding to the rabbit-node, which then can either further all messages to one single computer as a part of a conversation/user-session, or interleave between the available workers. Here's the manual. As the video is encoded, it is streamed asynchronously over the message bus back. According to intel tests the bus itself should work well at high throughputs, but they had to use the interleaved tcp channel mode (they tested on a gigabit lan). Other users here have suggested FFlib. You might also look into having the workers convert into webM, but if FFlib works, that might be a lot easier. Each worker publishes over AMQP the next encoded video piece. A server-running program, e.g. the router program I talked about before, starts sending to the client (see no. 4)
4. Have a client-program, e.g. silverlight/flash connect (for example over the same socket that you opened for client->server data, or over HTTP), and read the byte-stream with a decoder. Render the output.

Answer 5

Take a look at Video.Show by Vertigo. It's an open source website for user-generated video content. It uses the Expression Encoder to handle compression/video editing. It's not exactly what you need, but it's a good start!

Answer 6

You could use Silverlight for capture as is mentioned above, and then use Expression Encoder to push it to a stream server or stream from there directly.

It should have everything you need:

Smart encoding/smart recompression for WMV if the source is also WMV and no frame operations are performed [4], cuts editing, serial batch encoding, Live encoding from webcams and DV camcorders

Decoding/import format support because of DirectShow

Smooth streaming (720p+ video using HTTP) with optimized client (Silverlight) and server (IIS with smooth streaming)

WebDAV publishing, publishing plugins for Silverlight Streaming, Amazon S3

Importing XAML overlays created in Expression Design and customizing their timing, animation, opacity, placement and looping

JavaScript trigger events

Windows Media 11 SDK and VC-1 SDK integration, native MPEG-2 decoder

Adding captions to videos using SAMI or W3C Timed Text format

Previewing and comparing encoding settings in real time

Screen capture

Object model for the encoding engine, SDK downloadable separately