This question is alarmingly similar to those asking how to write an encryption algorithm. The answers to both should end in gentle reminders about industry standard solutions that already:
- embody years of experience and constant improvement,
- are probably far more secure than any home-grown solution, and
- account for ancillary requirements, such as efficiency.
A firewall must inspect every packet efficiently and accurately, and it therefore runs within the OS kernel or network stacks. Errors or inefficiencies jeopardize the security and performance of the entire machine and those downstream.
Building your own low-level firewall is an excellent exercise that will provide an education across many technologies. But for any real application, it's much safer and smarter to build a shell around the existing firewall API. Under Windows, the netsh
command will do this; Linux uses netfilter
and iptables
. Googling any of these will point you to lots of theory, examples, and other helpful information.
So, to get started, I'd brush up on TCP/IP (specifically, the header information: ports and protocols), then learn about the various types of attacks and how to detect them. Learn about each operating system of interest and how it interacts with the network stacks. Finally, think about administration and logging: how will you configure your firewall and trace packets through it to ensure it's doing what you want it to do?
Good luck!