I want to install a new system call in a place of an unused one and wait for instructions from userspace. i dont know how to do it.
Assuming you are talking about Linux 2.6, you should look at this guide: http://www.ibm.com/developerworks/linux/library/l-system-calls/index.html
More information:
http://tldp.org/HOWTO/html_single/Implement-Sys-Call-Linux-2.6-i386/
Note that you cannot install system calls in a module - the kernel image must be recompiled.
Assuming you're discussing Linux...
Mucking with the system call table is a bad idea. Kernels have some security defenses against changing it dynamically (which is a good thing), so you'll have to rebuild your entire kernel to make the modifications.
Here's a better approach that's just as flexible. Build a module that creates a new block device and implements your "system call" as an ioctl on that device. You won't have to recompile the kernel to implement this, and you won't have to worry about touching assembly files. When you're doing iterative development, recompiling and reinserting a module lets you work much faster than if you needed to reboot for your changes to take effect.
Look at Linux Device Drivers, Ch5 for info on writing ioctls.