Java for Audio Processing is it Practical?

Question

Is Java a suitable alternative to C / C++ for realtime audio processing?

I am considering an app with ~100 (at max) tracks of audio with delay lines (30s @ 48khz), filtering (512 point FIR?), and other DSP type operations occurring on each track simultaneously.

The operations would be converted and performed in floating point.

The system would probably be a quad core 3GHz with 4GB RAM, running Ubuntu.

I have seen articles about Java being much faster than it used to be, coming close to C / C++, and now having realtime extensions as well. Is this reality? Does it require hard core coding and tuning to achieve the %50-%100 performance of C some are spec'ing?

I am really looking for a sense if this is possible and a heads up for any gotchas.

Answer 1

Sure, why not?

The crucial questions (independent of language, this is from queueing theory) are:

• what is the maximum throughput you need to handle (you've specified 100 x 48kHz, is that mono or stereo, how many bits equivalent at that frequency?)
• can your Java routines keep up with this rate on the average?
• what is the maximum permissible latency?

If your program can keep up with the throughput on the average, and you have enough room for latency, then you should be able to use queues for inputs and outputs, and the only parts of the program that are critical for timing are the pieces that put the data into the input queue and take it out of the output queue and send it to a DAC/speaker/whatever.

Delay lines have low computational load, you just need enough memory (+ memory bandwidth)... in fact you should probably just use the input/output queues for it, i.e. start putting data into the input queue immediately, and start taking data out of the output queue 30s later. If it's not there, your program is too slow...).

FIRs are more expensive, that's probably going to be the bottleneck (& what you'd want to optimize) unless you have some other ugly nasty operation in mind.

Answer 2

For an audio application you often have only very small parts of code where most of the time is spent.

In Java, you can always use the JNI (Java Native interface) and move your computational heavy code into a C-module (or assembly using SSE if you really need the power). So I'd say use Java and get your code working. If it turns out that you don't meet your performance goal use JNI.

90% of the code will most likely be glue code and application stuff anyway. But keep in mind that you loose some of the cross platform features that way. If you can live with that JNI will always leave you the door open for native code performance.

Answer 3

There certainly are audio applications (for example see) written in Java so I'll offer that as a proof by existence. I'm not saying it's trivial but I don't see any reason why Java couldn't perform on par with C/C++ for this.

Answer 4

One thing I didn't see in your question is whether you need to play out these processed samples or if you're doing something else with them (encoding them into a file, for example). I'd be more worried about the state of Java's sound engine than in how fast the JVM can crunch samples.

I pushed pretty hard on javax.sound.sampled a few years back and came away deeply unimpressed -- it doesn't compare with equivalent frameworks like OpenAL or Mac/iPhone's Core Audio (both of which I've used at a similar level of intensity). javax.sound.sampled requires you to push your samples into an opaque buffer of unknown duration, which makes synchronization nigh impossible. It's also poorly documented (very hard to find examples of streaming indeterminate-length audio over a Line as opposed to the trivial examples of in-memory Clips), has unimplemented methods (DataLine.getLevel()... whose non-implementation isn't even documented), and to top it off, I believe Sun laid off the last JavaSound engineer years ago.

If I had to use a Java engine for sound mixing and output, I'd probably try to use the JOAL bindings to OpenAL as a first choice, since I'd at least know the engine was currently supported and capable of very low-latency. Though I suspect in the long run that Nils is correct and you'll end up using JNI to call the native sound API.

Answer 5

I think latency will be your major problem - it is quite hard to maintain latency already in C/C++ on modern OSes, and java surely adds to the problem (garbage collector). The general design for "real-time" audio processing is to have your processing threads running at real time scheduling (SCHED_FIFO on linux kernels, equivalent on other OSes), and those threads should never block. This means no system calls, no malloc, no IO of course, etc... Even paging is a problem (getting a page from disk to memory can easily take several ms), so you should lock some pages to be sure they are never swapped out.

You may be able to do those things in Java, but java makes it more complicated, not easier. I would look into a mixed design, where the core would be in C, and the rest (GUI, etc...) would be in java if you want.

Answer 6

Why not spend a day and write a simple java application that does minimal processing and validate whether the performance is adaquate.

Answer 7

Check out a library called Jsyn.

http://www.softsynth.com/jsyn/

Answer 8

Java is fine for many audio applications. Contrary to some of the other posters, I find Java audio a joy to work with. Compare the API and resources available to you to the horrendous, barely documented mindf*k that is CoreAudio and you'll be a believer. Java audio suffers from some latency issues, though for many apps this is irrelevant, and a lack of codecs. There are also plenty of people who've never bothered to take the time to write good audio playback engines(hint, never close a SourceDataLine, instead write zeros to it), and subsequently blame Java for their problems. From an API point of view, Java audio is very straightforward, very easy to use, and there is lots and lots of guidance over at jsresources.org.