views:

476

answers:

9

I'm working with Java project that requires very advanced manipulations of images. In fact, I'm doing most of the manipulation using OpenCV, and I'm using JNI to wrap around the OpenCV functions that I need. I am extremely satisfied with the performance OpenCV gives, the people who wrote the OpenCV code deserve great great credit for the code. In sharp contrast to what I experience with the code Java devs wrote.

I started out optimistic over the choice of my programming language, my first working iteration of the project works fine, but its performance is nowhere near to realtime (getting about 1 frame per 2 seconds.) I've done some optimizations of MY code and its helped a lot. I've been able to push the frame rate up to about 10-20 frames per second, which is great, but what I'm finding is that to do any further optimizations I have to rewrite Java code to do the same thing but 10-20x more efficient.

I'm appalled at how the developers of Java pay very little attention to performance, especially when writing the classes for Media related classes. I've downloaded OpenJDK and I'm exploring the functions I'm using. For example, there is a function under the Raster class called getPixels(...) and it gets the pixels of the image. I was expecting this function to be a highly optimized function in the source code, with several calls to System.arrayCopy to further optimize performance. Instead what I found was extremely "Classy" code, where they are calling 5-6 different classes and 10-20 different methods just to accomplish what I can do in one line:

for (int i =0; i < n; i++) {
  long p = rawFrame[i];
  p = (p << 32) >>> 32;
  byte red = (byte) ((p >> 16) & 0xff);
  byte green = (byte) ((p >> 8) & 0xff);
  byte blue = (byte) ((p) & 0xff);
  byte val = (byte)(0.212671f * red + 0.715160f * green + 0.072169f * blue);
  data[i] = val;
  grayFrameData[i] = (val & 0x80) + (val & (0x7f)); 
}

The code above transforms an image to grayscale and gets the float pixel data, in roughly 1-10ms. If I wanted to do the same with Java built in functions, the conversion to grayscale itself takes 200-300ms and then grabbing the float pixels takes about 50-100ms. This is unacceptable for real time performance. Note to get a speedup, I make heavy use of bitwise operators, which Java devs shy away from.

I understand that they need to handle the general case, but even so, can't they at least give options for optimizations or at the very least a warning how slow this code may perform.

My question is, at this late point in the development (I already have my first iteration, not I'm working on a second that performs more in real time) should I bite the bullet and switch over to C/C++ where I can fine tune things a lot more, or should I stick with Java and hope things will become more realtime friendly so that I won't have to rewrite already implemented Java code to get a speedup.

I'm really beginning to become disgusted with how "classy" and slow Java really is. The amount of classes there are seems like overkill.

+3  A: 

My suggestion would be it depends on how important the manipulation of images is compared to the project as a whole, and relative to whatever advantages java does bring. Clearly you can write fast code in java (as you demonstrated) if you need to. However, if 80% of your project is going to consist of such optimization, I would certainly rethink Java as the language choice here.

On the other hand, if this represents 20% of the application, with the other 80% being the user functionality around providing this conversion, then perhaps having to do the work to get the manipulation done is a worthwhile trade off to not have to deal with your own memory management, and to have whatever other APIs java is giving you for user interaction (Web, Swing, SWT, whatever you are using).

Java is not known for its real time abilities due to the garbage collector. That might come to bite you as well, so be careful about that.

Yishai
It's clear from the OP that the image manipulation is a very important part of the application and it seems as a nonfunctional requirement to have real time image manipulation
hhafez
@hhafez, it is not at all unusual for the core important functionality to be 20% of the overall project, the rest being about user interaction and business logic that leads to the core purpose.
Yishai
+6  A: 

My question is, at this late point in the development (I already have my first iteration, not I'm working on a second that performs more in real time) should I bite the bullet and switch over to C/C++ where I can fine tune things a lot more, or should I stick with Java and hope things will become more realtime friendly so that I won't have to rewrite already implemented Java code to get a speedup.

You are asking should I

  1. Switch to a language where I can satisfy my performance requirements.
  2. Stick with Java and hope things improve.

There might be other options.... but option 2 doesn't seem realistic, you can't just "hope" that the code becomes faster :p

A few points to note:

  1. OpenJDK does not nescceraly have the same performance as the Sun JDK, have you tried the Sun JDK?
  2. If the performance optimizations that you need done are in a few methods, then it might be worth re-writing them and sticking with Java...
hhafez
A: 

Premature optimization is the root of all evil.

Rather than complain, write an optimized set of libraries and release them, but it would just be wrong to create a "Reference" java implementation that was pre-optimized to some non-existent target.

The main point of a reference implementation is to make understandable, maintainable code--it has to be. I think that there was always an expectation that where necessary vendors would analyze this understandable version and re-implement parts for speed.

Bill K
who said he is optimizing prematuarly?
hhafez
@hhafez, I think Bill's point is about why the standard libraries are not optimized the way the poster would like, not about this particular use case.
Yishai
@Yisha thank you, yes. As indicated by my point that he should write a set of optimized libraries. There is a place for it, but only when needed--the place for that is not the reference implementation.
Bill K
A: 

In addition to what others have said, you can contribute optimizations to the JDK. If you can provide a powerful optimization that doesn't sacrifice generality or readability, I expect you'll be able to get your patch included in a future JDK release.

Thus, you don't have to hope the JDK can become better. You can help make it happen.

Eddie
A: 

As I understand it, the very latest versions of Java (or maybe it was JavaFX) have methods that allow you to access advanced functionality in the Video hardware of your system. I'm sorry I am being so general, I believe I heard about it on Java Posse and since I'm stuck in Java 1.3 land I never really had a chance to check it out--but I do remember hearing something like that.

Here's something about it: But it looks like it will only be in Java 7 :(

Also looks like it will only support playing a stream and rudimentary stream manipulation at first--but maybe the "Wait around and Java will improve" approach might actually work.

Bill K
A: 

What's stopping you from writing an optimized version of the methods you wish to use instead of using the built-in methods? If that's not possible, why not write your objects in a more native language, and import that into your existing application?

SnOrfus
+1  A: 

I don't know how much of a performance boost you'll get, but if you have a long-running process doing repetitive things you should try to run the Server Hotspot VM using java -server . It performs much better than the client VM that's the default on Windows, which is optimized for fast startup time.

ykaganovich
+11  A: 

I've done computer vision work with Java, and I may get downvoted for saying this, but it is perfectly usable for computer vision and realtime stuff, you just have to know how to use it.

Potential Optimizations:

If you need help optimizing your code, I'd be glad to assist -- for example, I can tell you that you will probably get a performance boost by making a method

`public static final int getGrayScale(final int pixelRGB){
    return (0.212671f * ((pixelRGB >> 16) & 0xff) + 0.715160f * ((pixelRGB >> 8) & 0xff) + 0.072169f * ((pixelRGB) & 0xff));
}`

and using this in your for{pixels} loop. By using a method call, the JVM can much more heavily optimize this operation, and can probably optimize the for loop more too.

If you've got RAM to burn, you can create a static, final lookup table of output grayscale bytes for all possible 24-bit pixel pixel colors. This will be ~16 MB in RAM, but then you don't have to do any floating point arithmetic, just a single array access. This may be faster, depending on which JVM you are using, and whether or not it can optimize out array bounds checking.

Places to find similar, faster image processing code:

I would strongly suggest that you take a look at the code for the ImageJ image processing app (can't link thanks to StackOverflow being retarded) & its libraries, specifically ij.process.TypeConverter. Just like your code, it relies heavily on direct array operations with bit-twiddling and a minimum of extra array creation. The Java2D libraries (part of the standard JRE) and the Java Advanced Imaging(JAI) library (can't link thanks to StackOverflow being retarded) provide other ways to do image processing directly on image data rapidly without having to roll your own operation every time. For Java2D, you just have to be careful which functions you use.

Why the Java2D libraries are so indirect:

Most of the "class-iness" is due to supporting multiple color models and storage formats (I.E. HSB images, float-based color models, indexed color models). The indirection exists for a reason, and sometimes actually boosts performance -- the BufferedImage class (for example) hooks directly into graphics memory in recent VMs to make some operations MUCH faster. Indirection lets it mask this from the user a lot of the time.

BobMcGee
A: 

Not sure if this is the appropriate place, but we can profile your Java code and compile your code ahead-of-time to native machine code based on your configuration and control. This bypasses the JIT issue and any startup issues. In addition our realtime garbage collection technology will give you deterministic smooth performance. We fully support you for multicore multi-threaded applications as well. Our engineers specialize in optimizing graphics performance for Java with OpenJDK. We regularly get C++ -like performance out of Java applications.

dbeberman