is it possible to create java application that will work as background process on symbian smartphones?
You can approximate it but J2ME (the version of java on mobile phones) may not be the right technology to do this.
starting a MIDlet (a Java application for mobile phones) when the phone is switched on is tricky at best without coding a small Symbian OS C++ module that will start it for you. If you want to try anyway, look at the PushRegistry class in the MIDP specs (http://java.sun.com/javame/reference/apis/jsr118/). The Content Handling API might provide some way to do it too (http://java.sun.com/javame/reference/apis/jsr211). When you are ready to give up, do it in C++.
Backgrounding a MIDlet isn't hard. The phone's "menu" key will do it for you. Programatically, Canvas.setCurrent(null) has a good chance of working. Trying to trick the phone by providing a fully transparent GUI and not handling any keypad activity will absolutely not work. Creating and starting a separate Thread in the MIDlet should allow you to keep something running even after your overload of MIDlet.pauseApp() has been called by the application management system.
The real issue is that the MIDlet will not have any Inter Process Communication system unless you make one. The usual way of doing that is a loopback socket connection over which you transfer data. Not a nice or efficient way of simulating IPC. Sharing a RMS record can only be done from within the same MIDlet suite (you can package several MIDlets into the same .jar file), I think. The code to create a provider/consumer data flow over a file connection is even uglier and will raise security issues.
Without any more information about what you want to use it for, my answer is : maybe but you should probably not try.
You will have in-built MIDP support for background MIDlets in MIDP 3.0 (http://jcp.org/en/jsr/detail?id=271). Don't hold your breath for devices to appear, however - might be some time. (Note that a few Symbian OS devices have more than just MIDP - the S-E p990 for instance, https://developer.sonyericsson.com/site/global/products/phonegallery/p990/p_p990.jsp).
As already pointed out, it might be helpful to have more information on what product functionality you are trying to implement - often more than one way to skin a cat.