I presume that you are running VIM in terminal and talk about VIM's visual selection, not selection with the mouse in terminal window.
Since in your case VIM isn't aware of the X, it obviously cannot communicate your selection to the X Window System. If you select something in terminal window with mouse, then you are selecting not in VIM, but in the terminal emulator which is aware of the X and communicates properly the current selection to X's clipboard.
You can try to play around with xsel
tool which allows one to access the X clipboard from command line. You would need to override the usual clipboard shortcuts to not only put the content of the visual selection into the usual VIM register(s), but also to pipe it to the xsel so that it becomes the new X selection. (Strangely enough, I do not have the xsel installed on the SLES10 system I have right now at hand.) That obviously would only work if (1) the terminal is running locally or (2) remote server you are connected to allows the X forwarding. If the X forwarding is disabled on the server where you edit the files, then you have no chance to do in VIM: only selecting with mouse in terminal window would work.