I wrote this C program and ran on my Windows system. My system hanged and not even Task manager was opening. Finally, I had to reboot. This is clearly a malicious program, but my antivirus does not detect this. Why?
#include<unistd.h>
main() {
while(1)
fork();
}
Antivirus programs don't recognize malicious behavior - they recognize patterns of know viruses that are already in the wild (file names, process names, binary signatures, etc.).
This is why they can often be subverted since they are a reactive solution to an evolving problem.
Developers don't typically use AV software due to the huge speed penalty, or at least they disable it on the filesystem subtree they work in.
But even so, that isn't the sort of pattern AV software tries to detect. The AV software looks for files you are reading and writing and changes to system state, or specific identified viruses or their prior identified signatures.
And how would it decide, anyway? From the point of view of a program there would be a fine line between an overloaded web server and a fork bomb.
Finally, this sort of behavior is kind of self-correcting. If we really had viruses arriving with nothing more damaging than a fork bomb we might just declare victory and say "don't run that".
BTW, did you run the fork bomb as administrator?
Well, it is not malicious behavior, looks more like a logic error in your code. I wish there will be an antivirus one day that could detect applications, drivers, ms updates, ms products that cause BSOD's =)
Your program is not a virus, because it cannot spread itself, that is, it can't infect other files/computers