Cross platform multimedia kiosk

Question

My team is tasked with building a full screen, kiosk-style application for playing back media files. Initially we need to support WMV / MP4 as well as some images in full 1080p, although down the line we will need to extend this to cover other formats (different videos formats as well as display of HTML, SWF, etc).

The application also contains a decent chunk of business logic relating to scheduling, logging, performance monitoring as well as network code to talk to a central server through web services (or maybe TCP) and potentially act as a server itself.

For our WMV / MP4 video playback, hardware acceleration will be a massive bonus. The targetted hardware has weak CPUs but strong graphics cards.

Here's the kicker: we're a .NET shop (our existing application is a WinForms smart client) and extremely experienced and productive in C# and the .NET stack. The app will initially be targetting Windows Embedded (.NET 3.0), but we will quickly need a Linux version as well. Between us we have some C/C++ experience and some Linux experience but we do not anticipate good productivity on that platform.

So I am soliciting recommendations specfically on the following points:

1. Video. On Windows we have seen good success using DirectShow.NET. On capable hardware, the WPF MediaElement also seems to perform well. What should we be using on Linux? libavcodec seems like a common choice. Is it hardware accelerated on NVidia graphics cards on Linux? What other options do we have on Linux? Is there something cross-platform that I could consider?

2. Stack. a) Ideally we could write the whole thing in .NET and then run under Mono on Linux. The video playback and presumably some other components (like performance monitoring) would not be supported on Mono. I guess we could rewrite these elements in, say, C++; but I'm guessing that most stuff on the business logic side would work. b) Maybe it's better to forfeit our up-front productivity on the Windows version for something that's cross platform out of the gate. What about Java? Do we have different options when it comes to video there? How about another framework? Something like QT? Can anyone else suggest something cross platform that would be relevant?

Broadly speaking, given the requirements, what would you use?

I appreciate any anwsers you might have.

Answer 1

One option would be to use Silverlight, and explore Moonlight as an option for the linux version. My understanding is that Moonlight has several media/codec plugins (I believe ffmpeg is the main provider) and can additionally use the MS codec pack to give you support for things like WMV/MP4.

Answer 2

You can use ffmpeg in mono and .net. This may or may not include video display - ffmpeg usually just provides you with a decoded bitmap that you can do whatever you want with, be it display it in a window, save it in a file, whatever. If you use ffmpeg-sharp the same code should work on Windows or Linux. Really, putting the bitmap in a window is the easy part.

Answer 3

My suggestion is that you use Fluendo's GStreamer components for the video playback as it has support for hardware acceleration where available and fully licensed codecs.

You can look at the Banshee media player which support video playback if you have the Fluendo/GStreamer packages installed. Get OpenSUSE 11.2 which contains everything you need to try it and develop, and then buy and install the Fluendo codecs.

Source code wise, Banshee does the video display from C#, look here:

The C# source code consuming GStreamer and doing the video rendering is here:

http://git.gnome.org/browse/banshee/tree/src/Extensions/Banshee.NowPlaying/Banshee.NowPlaying

The C supporting library to call into Fluendo is available here:

http://git.gnome.org/browse/banshee/tree/libbanshee

For testing Banshee, you do not need to buy anything, but your video codecs will be limited to Ogg/Theora encoded videos. Once you get Fluendo's codecs you will be able to play WMV files.

Answer 4

Moonlight offers two codecs: (a) A fully licensed version that comes straight from Microsoft and requires no further negotiation with the MPEG-LA and other patent holders, or (b) an ffmpeg backend that requires you to negotiate with the patent stock holders if you plan on using.

You could build a Silverlight-based application, the trick to get access to the local system is very simple: you run a local web server that exposes those services.

You can still use C#/Sqlite or VistaDB as your storage system as part of your Silverlight application.

You could host the silverlight app in http://localhost/App.xap and this app would gain local access to the machine by contacting a REST or SOAP web service on http://localhost/rest.ashx or http://localhost/soap.asmx

For example, if you needed to read some values from a scanner connected to the machine, you would issue this request:

http://localhost/scanner.ashx?operation=scan_badge

Then your scanner.ashx HttpHandler will do the actual scanning (this one has full system rights) and return the value to the Silverlight application.