recentry we had task to need big amount of memory (RAM) but have hot enought...
but we have much free HDD space
maybe exists tools (linux) which can handle process memory request (etc) and have limit RAM swap it into HDD (like 'nice' tool handle priority)?
What do you mean by "much"? If "much" is more than a handful of GB:s, you need to be on 64-bit hardware and operating system, in order for a process to be able to address that large an amount of storage.
I'm pretty sure you can use virtual memory on 64-bit Linux at least, to provide apps with more storage than there is RAM. Of course, it won't be fast if the access pattern is random.