I would certainly recommend using well tested libraries if possible. Especially for the SSH side of SFTP - as mistakes in implementation could lead to security vulnerabilities.
That said, you seem like an experienced person, and you may find that none of the libraries out there (SharpSSH etc.) are appropriate, you should consider seperating your network implementation out into its own library and open sourcing it for two reasons:
- If it is useful to you, it may well be useful to someone else.
- You might get some helpful code review
Given all that, other guidelines to follow would be similar to other developments, but perhaps with more rigour regarding security and testing. Making sure you run static analysis and that you unit test for failure cases.
Also: Use, but don't trust the lower layers of the OSI model that are provided for you. If there is a scenario that breaks your code, you can bet that the network will find it eventually.
Hope this helps.