As others have said, this is standard java behaviour and shouldn't really be worked around.
You are probably better off creating another folder outside of the source folder.
However there is a way to do it in Eclipse, be warned that this might come back to haunt you if you later try to build outside Eclipse as other tools won't honour these settings.
You can exclude a folder from the source path by opening the project properties.
Selecting Java Build Path->Source
Select the appropriate source folder and select the Excluded: child item
Hit the Edit... button.
Select the Add button next to the Exclusion patterns: pane
Add your folder and Finish/OK back to the editor.
The matching resources will now be excluded from compilation within Eclipse. It will still appear as a package in the editor though.