Can be embedded in Java using the OOoBean. The original code runs under Swing but you can make it work under SWT, too, using the AWT/SWT Bridge (example code).
The project is open source (LGPL). The code is OS dependent but there are versions available for all major OSs (Windows, Linux, MacOS and Solaris are supported on the project's site but you can download the source and compile it for others, as well).
Since the underlying product is a full blown office application, it offers all the features you might want (fancy styles, online spell checking in different languages, embedding pictures, flow text around pictures, you name it).
The drawback is that you must have OpenOffice installed; in fact, the editor is not part of Java but it's a remote process which just renders in your Java app's window. This means that start up time is quite noticeable and performance can be sluggish. Also handling is sometimes "odd". It takes some configuration to get rid of additional borders and unwanted document pagination.
Because of this, the editor is not really extendable from Java. You will either have to write code in C++ or using OOo's own built-in language. What you can do is you can register listeners in the document and you have full access to the underlying model for the document. This way, you can hook into the important parts and modify the behavior but this often feels like performing keyhole surgery.
My conclusion: If OO has the features you need, this might be the most simple way to get what you need. If you need a lot of customization, you'll fight an upstream battle against the framework's defaults.