No, I don't think it will hurt you. I've been doing systems programming (drivers, kernel, embedded) for about five years, but I got my start writing C++ application software.
Take the next year or two to do the following:
1) Build up some base skills in the industry: version control, software architecture, learning to read APIs, et cetera. You'll use these in any decent software job.
==> If you want to work in the Windows world, take the year to learn how to make C# call and work with Win32/C/C++. Most people with experience in the industry don't know much about .NET, but application programmers these days only want to use .NET and no longer want to call raw DLLs.
2) Make sure your OS knowledge C programming skills are up to speed: pointers, virtual memory, et cetera. Read books on the Windows Internals or Linux/BSD kernels.
3) Get some relevant experience in your spare time and learn how to use the tools: kernel debuggers, kernel modules, building kernel modules, etc. You can buy demonstration hardware very cheap (US $100~$200) and then learn how to write your own Linux kernel modules or Windows device drivers. Sure, the hardware might be trivial, but you'll be learning how the OS interacts with the hardware and that is the key.
==> I think this is key if you want to work with Linux and Mac technologies.
4) Consider your industry goals. Are you set on the PC/Server industry? There is a lot of competition in that market, and in Taiwan and China there are plenty of entry-level systems programming jobs where so much of the commodity hardware and computers are produced. However, if you're open to working in the telecom, defense or aerospace industries, there are lots of opportunities to do systems level work, including compilers for customer hardware and what not.