Flash Media Server: Trouble with recording 2 audio/video streams at the same time?

Question

In short

For a project I need an audio/video-chat for 2 people, with the ability to record (part of) the session. I am running into issues where the 2nd user's recorded video gets messed up, with massive amounts of (seemingly) skipping frames and/or audio loss, most likely caused by the audio stream (when not recording the audio, the problem doesn't appear to occur).

Overview

For a project I need a setup where two clients can video(+audio, obviously :) chat with one another. Also, the 'host' should be able to record the session when (s)he presses a button, and stop recording in the same way.

While this setup is far from rocket science, I've been experiencing issues that I can't seem to figure out. The clients connect fine, and each other's videos show up propely and they can even hear each other just fine. I chose to re-initialize the connection when the host starts the recording, which also works just fine. The recorded files, however, at times experience issues.

The possible cause

Usually the hosts' recording is great, with audio and video working as it should. The client video shows problems, with the video literally jumping when played back (in a number of applications such as VLC, KMPlayer, Adobe's own media player, etc.). I tried to debug the situation and it seems to be directly related to the audio, though how this happens I'm not sure. When I did tests with no recorded audio, both videos played back fine.

The problem

I'll try to explain what happens in the client video; the video plays back normally, with audio working fine too. But in seemingly random locations of the video, playback suddenly skips seconds of the video (and audio), so a one minute video sometimes lasts mere seconds (even though the playback bar and the such show that the video in fact lasts for a minute).

I have not found any logic in how it skips --some videos show only 4/5 gaps of a few seconds each, others jump 20-30 seconds ahead--, so I'm assuming it's a random thing.

Scenarios

I have tested multiple scenarios, and the problem seems consistent enough (as in, it occurs every time, just not when I dont record audio). I have used a local (developer edition) FMS server, the hosted influxis service, and I have used two local computers, one local machine (using 2 webcams) and 1 local, 1 (really) remote computer. All setups seem to have the same issues with the recorded (client) video.

In closing

I'm not sure what details you need, so please, ask me for anything you might need to help me find a solution to this. I have searched and debugged like there was no tomorrow, and haven't been able to figure out what is causing this.

Many, many thanks in advance!

-Dave

Answer 1

Dave, have you ruled out latency? are the computers with both cams connected to a high speed network? I would also check the processor utilisation on the machine which is recording.

Answer 2

Hi Dave,

I've had some pretty extensive experience with FMS and specifically with influxis... Also did recording of both audio and video as the system I had needed it for security reasons. However, in all cases the video and audio were recorded on the server - not on the same machine as either client.

The issues I've seen include a vast amount of tweaking of both audio and video quality - to find the correct mix to get optimal results. That's where I'd start, adjust quality down and see where that gets you.

For a test environment, I'd suggest using different machines so the CPU doesn't become your issue - as the actual environment would have the clients on different boxes with more CPU.

Contact me with addition information, etc - I and several friends have a system working and I'm happy to help.

<>

Looked over my notes - we were able to talk to influxis about the lagging audio - and there were a couple of server setting they played with AND it was our frames-per-second and other quality guys...

Answer 3

Hi there,

I'm also experiencing the very same issue. In my case, I'm trying to record 2, 3 or four videos at the same time. The problem seems to be random.

I've tested bandwidth, CPU utilization and some other variables, but none of them seems to be related to the problem.

Any ideas?

Answer 4

I'm not sure if my problem was the same as yours, but we found a solution.

The flash piece needed to set the silence level (setSilenceLevel) such that it didn't insert audio packets that signal no audio. These audio packets caused recording problems that produced random skips and audio drops.